Systems, methods, and bone mapper devices for real-time mapping and analysis of bone tissue

ABSTRACT

Apparatuses, systems, and methods for performing real-time analysis of bone tissue during a surgical procedure are disclosed herein. In some embodiments, the method includes receiving at least one measurement of at least one tissue sample from a hybrid multi-wavelength photoacoustic measurements (MWPM) component. The method can also include identifying one or more reference cases, from a plurality of reference cases, based on correlations between the at least one measurement and previous measurements in each of the plurality of reference cases. Once the reference cases are identified, the method can include determining at least one bone condition of the patient and sending the at least one determined bone condition to a computing device accessible by a surgeon. In some embodiments, the method also includes creating a three-dimensional (3D) map the tissue sample using the at least one measurement and sending the 3D map to the computing device.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.17/746,608, filed May 17, 2022, which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present disclosure is generally related to apparatuses, systems, andmethods for surgical analysis of tissue samples and, more specifically,to bone mapper devices for real-time mapping and/or analysis of bonetissue using multi-wavelength photoacoustic measurements.

BACKGROUND

More than 200 million surgeries are performed worldwide each year, andrecent reports reveal that adverse event rates for surgical conditionsremain unacceptably high, despite traditional patient safetyinitiatives. Adverse events resulting from surgical interventions can berelated to errors occurring before or after the procedure as well astechnical surgical errors during the operation. For example, adverseevents can occur due to (i) breakdown in communication within and amongthe surgical team, care providers, patients, and their families; (ii)delay in diagnosis or failure to diagnose; and (iii) delay in treatmentor failure to treat. The risk of complications during surgery caninclude anesthesia complications, hemorrhaging, high blood pressure, arise or fall in body temperature, etc. Such adverse events can furtheroccur due to medical errors, infections, underlying physical or healthconditions of the patient, reactions to anesthetics or other drugs, etc.Conventional methods for preventing wrong-site, wrong-person,wrong-procedure errors, or retained foreign objects are typically basedon communication between the patient, the surgeon(s), and other membersof the health care team. However, conventional methods are typicallyinsufficient to prevent surgical errors and adverse events duringsurgery.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example surgical system, inaccordance with one or more embodiments.

FIG. 2 is a block diagram illustrating an example machine learning (ML)system, in accordance with one or more embodiments.

FIG. 3 is a block diagram illustrating an example computer system, inaccordance with one or more embodiments.

FIG. 4A is a block diagram illustrating an example robotic surgicalsystem, in accordance with one or more embodiments.

FIG. 4B illustrates an example console of the robotic surgical system ofFIG. 4A, in accordance with one or more embodiments.

FIG. 5 is a schematic block diagram illustrating subcomponents of therobotic surgical system of FIG. 4A, in accordance with one or moreembodiments.

FIG. 6 is a block diagram of a system for performing real-time analysisof bone tissue in accordance with some embodiments of the presenttechnology.

FIG. 7 is a block diagram of the hybrid MWPM component communicativelycoupled in the system of FIG. 6 in accordance with some embodiments ofthe present technology.

FIGS. 8A and 8B are flow diagrams of a process for operating anoperating room in accordance with some embodiments of the presenttechnology.

FIG. 9 is a table illustrating an example operating room real-timeequipment database in accordance with some embodiments of the presenttechnology.

FIG. 10 illustrates an example of a three-dimensional map that can begenerated during real-time analysis of bone tissue in accordance withsome embodiments of the present technology.

FIG. 11 is a flow diagram of a process transferring information to anexternal equipment in accordance with some embodiments of the presenttechnology.

FIG. 12 is a table illustrating an example bone database in accordancewith some embodiments of the present technology

FIG. 13 is a table illustrating an example pathology database inaccordance with some embodiments of the present technology

The drawings have not necessarily been drawn to scale. Similarly, somecomponents and/or operations can be separated into different blocks orcombined into a single block for the purpose of discussion of some ofthe implementations of the present technology. Moreover, while thetechnology is amenable to various modifications and alternative forms,specific implementations have been shown by way of example in thedrawings and are described in detail below. The intention, however, isnot to limit the technology to the particular implementations described.

DETAILED DESCRIPTION Overview

An individual's tissue composition and skeletal structures changethroughout their life. For example, changes in bone masses and bonestructures (sometimes referred to collectively as “bone mass” herein)are generally characterized by a period of growth (e.g., linear growthof endochondral growth plates and radial growth from periostealapposition) and a period of slow loss as the individual ages. Loss ofbone mass during the latter period can lead to and/or be accelerated bya variety of medical conditions. For example, osteoporosis is adebilitating disease characterized by large decreases in skeletal bonemass and mineral density, structural deterioration of bones (e.g.,degradation of bone microarchitecture), and/or increases in bonefragility and susceptibility to fracture. Osteoporosis is often precededby clinical osteopenia, which is a condition defined by a bone mineraldensity that is below the mean value for bone mineral density for anindividual's age by between 1 and 2.5 standard deviations. Osteoporosisis one of the most expensive diseases to detect and treat, and isassociated with long-term residential care and lost working days whilean individual is treated. Early and accurate detection of changes to anindividual's bone mass and structures, for example during clinicalosteopenia, can drastically reduce the costs and lost time associatedwith treatment. However, osteopenia typically does not result in manyphysical symptoms of reducing bone density (e.g., pain, noticeableincreases in bone fragility, and the like). Further, although there area variety of techniques and chemical analyses available to calculatebone material density and detect changes in bone density, conventionaldetection techniques are often limited by cost, accessibility, anddangers associated with the detection techniques. As a result, it can bedifficult to confirm that an individual has osteoporosis even when theyare exhibiting physical symptoms, let alone anytime before then.

For example, conventional techniques for monitoring tissue compositionand bone masses (e.g., to detect and monitor osteoporosis) often beginwith a non-invasive assessment method based on the use of X-rays orultrasound to measure an individual's bone mass density (BMD). ButX-ray-based techniques use ionizing radiation, which is not ideal forpediatric and/or long-term, repetitive monitoring of an individual'sbone mass. Dual-energy X-ray absorptiometry (DEXA) is a relatedtechnique that uses two low-dosage X-ray beams to measure BMD. In a DEXAtechnique, X-ray beams with differing energy levels are aimed at thepatient's spine, hip, or whole body to take measurements. A computer canthen calculate the individual's BMD based on the fact that differentbones and/or bone densities absorb different energy levels. DEXAtechniques can be highly accurate, but they involve complex equipmentthat is bulky, expensive, and still require exposure to radiation.Furthermore, both X-rays and DEXA are limited by their operatingwavelength and are typically unable to detect important details aboutthe microstructure of a bone, organic matrixes associated with theindividual's bone, and the like.

Quantitative ultrasound (QUS) technology is another example that is arelatively low-cost alternative compared to the X-ray-based techniques.Because QUS methods are primarily based on the measurement of soundvelocity and broadband ultrasound attenuation through tissue, the QUSmethods avoid exposure to radiation. However, the specificity ofidentifying bone composition in QUS methods is limited when bonediseases are determined by microstructure and chemical changes in thebone. This limitation is significant because an individual's bone healthis dependent on not only the mass and structure of non-organic mineralmatrixes but also the organic matrixes associated with the individual'sbone blood flow and cellular metabolism. Furthermore, the QUS methodsrequire a long turnaround time to provide a complete analysis of thetissue or a biopsy sample.

The use of biochemical sensors is another option to help monitor bonemass. Biochemical sensors can provide real-time information from in situmeasurements. However, safely incorporating biosensors, such as acousticemission sensors, in monitoring bone masses requires careful surgicalimplantation and coupling of the sensors. For example, incorrect sensorplacement can result in damage to the bone masses sought to bemonitored. Further, surgical implantation can lead to infections andother adverse surgical-related complications. Additionally, thematerials necessary for biosensors are often expensive and/or restrictedby public health agencies.

Optical imaging of tissue is yet another option for monitoring bonemass. For example, conventional multi-wavelength photoacousticmeasurements (MWPM) generate a μ3 value that can be used to monitor andassess an individual's bone density and changes over time (sometimesalso referred to as multi-wavelength photoacoustic (MWPA) analysis).However, tissue is a highly scattering medium for electromagnetic wavesin the optical spectral range that imposes limits on the use of opticalimaging techniques. As a result, two optical imaging techniques havebeen developed. Ballistic (minimally scattered) techniques provide arelatively high resolution, but are limited to a low imaging depth inthe tissue (e.g., around 1 millimeter (mm)) that is imposed by theoptical diffusion limit. When incident photons reach this limit, most ofthem have undergone tens of scattering events, which scramble the photonpaths and inhibit effective optical focusing. In contrast, diffuseoptical tomography techniques can probe greater distances (e.g.,centimeters) into tissue, but with relatively low resolution (e.g.,resolution equal to ⅓ of the imaging depth). One challenge of thesetechniques is that the randomized paths of diffuse photons render imagereconstruction mathematically ill-posed. An additional limitation toeach technique is that the bone analysis from optical imaging does notprovide an on-site chemical composition of the bone tissue and,therefore, requires additional time to perform analysis of the sample(e.g., via a biopsy). Further, the Ballistic imaging elements anddiffuse optical tomography imaging elements can be positioned atdifferent locations and/or orientations when imaging tissue.Unfortunately, the resulting varying trajectory of the acousticemissions, in conjunction with ill-posed positioning between theBallistic and diffuse optical tomography imaging elements, can lead toinconsistent interpretation of the optical output. Still further, it maybe difficult for a surgeon to visually identify features of the analyzedtissue during a surgical procedure. For example, a diffuse opticaltomography imaging element may be used to help identify diseased bonetissue to be excised, but the surgeon (or robotic surgery apparatus) maybe unable to determine, for example, margins of the diseased tissueidentified by the diffuse optical tomography imaging element. This canlead to inaccurate excisions, inaccurate biopsy diagnoses resulting fromanalysis of incorrect bone tissue, and can complicate surgical steps.

To overcome one or more of these technical deficiencies in conventionalmethods, systems, and apparatuses disclosed herein include a tissuemapper device configured to address the shortcomings of the methodsdiscussed above. For example, the tissue mapper device can include amultiplexing imaging system that can take multiple types of measurementsand correlate the measurements with known cases in order to address theshortcomings of the methods discussed above. Additionally, oralternatively, the tissue mapper device can capture image data of tissueanalyzed by the multiplexing imaging system to generate one or morethree-dimensional (“3D”) maps of, for example, anatomical element,target tissue, etc. The 3D map(s) can be generated in real-time toassist with surgical procedures and/or provide a real-time diagnosis ofthe target tissue.

The multiplexing imaging system can include a hybrid imaging componentthat is configured to perform multiple types of measurements from asingle device. For example, the multiplexing imaging system can includea hybrid or multisampling MWPM component or unit (sometimes referred toherein as the “hybrid MWPM component” and/or the “hybrid MWPM unit”)configured to perform multiple types of multi-wavelength measurements.For example, the hybrid MWPM component can generate a BMD value, MWPMvalue, μ_(a) value, and/or various other values of the same region. Oneresult is that the hybrid MWPM component, and the associated systems andmethods, can provide an accurate, real-time diagnosis of an individual'sbone condition (e.g., bone mass, bone density, normal bone, cancerousbone, osteoporosis, clinical osteopenia, and the like). Another resultis that the hybrid MWPM component, and the associated systems andmethods, can generate a 3D map of the individual's bone in real-time andpresent the 3D map to a surgeon, doctor, or other medical professional.

In various embodiments, the multiplexing imaging system can includevarious other hybrid imaging components in addition to (or inalternative to) the hybrid MWPM component. By way of example, themultiplexing imaging system can include a hybrid x-ray component (e.g.,configured to take multiple x-rays at varying wavelengths that arereflected differently by tissue). In various embodiments, themultiplexing imaging system can include an acoustic component (e.g., anultrasound and/or MWPM component), an x-ray component, a CT scancomponent, and/or a component for any other suitable wave-basedmeasurement. As discussed in more detail below, these components can becombined in a single device (e.g., thereby taking measurements from asingle, known position) and/or from multiple devices with a well-definedposition and/or orientation with respect to each other. For example,each of the imaging components can be integrated into an end-effector ona robotic surgical apparatus to take the measurements. In anotherexample, one or more of the measurements can be taken by a devicepre-operation and integrated with other measurements. In a specific,non-limiting example, imaging from an x-ray taken pre-operation can becombined with ultrasound and/or MWPM measurements taken during anoperation.

Advantageously, the multiplexing imaging system can analyze the tissuefrom a single imaging reference position or range of known positions(e.g., multiple positions with known relative positions to create awell-posed relationship) to enable position-independent correlating ofthe values. In single imaging reference position embodiments, outputfrom the multiple sensing elements can be directly combined to provide acomposite analysis. For example, the system can select and process(e.g., weight, filter, etc.) output from one or more of the multiplesensing elements. The processed output can then be combined with output(e.g., image data, images, video, etc.) from any other devices (e.g.,imaging devices, cameras, X-ray machines, and the like). In embodimentsin which the multiple sensing elements are imaging elements, theoutputted data can be quickly combined via the well-posed problem toprovide a composite analysis. For example, relative positions betweenthe sensing elements can be stored by the system. The system can thenprocess the data (e.g., transform data, modify or scale data, etc.) toprovide for enhanced interpretation by a physician. Systems can storetransformation matrices that can be used to combine outputs from sensingelements located at different positions during tissue analysis.Advantageously, the transformation matrices allow for accurate analysisthe same tissues, features, or the like using multiple sensing elements.Further, the resulting composite analysis can then be overlaid ontoimage data (e.g., 3D renderings, topological maps, pictures, video, orother image data) to produce a diagnostic image or map (sometimesreferred to herein as a “composite image”). The composite image can beannotated by a user, a system programmed for annotation, etc. tofacilitate user review.

The methods disclosed herein can then combine the various measurementstaken by the multiplexing imaging system to take advantage of thestrength of various techniques and to generate a plurality of nodes(e.g., in the case of the hybrid MWPM unit, the BMD value may accuratelymeasure an amount of non-organic material, the MWPM value can accuratelyassess bone matrixes, and the absorption spectrum (μ_(a)) value canaccurately measure organic material). The resulting values can beselected and correlated to generate one or more multiplexed measurementsbased on, for example, patient information (e.g., age, condition, etc.),accuracy scores for the individual values, machine-learning models,and/or various combinations thereof. In some embodiments, themultiplexing imaging system includes multiple hybrid imaging componentspositioned and configured to direct acoustic energy at the same volumeor area of tissue. The methods disclosed herein can then correlate thecombined measurements to reference cases to identify similar individualswith known bone conditions, then the reference cases (and the combinedmeasurements) to diagnose the individual's bone condition. Accordingly,the systems and methods disclosed herein provide an accurate, real-timeassessment of the individual's bone condition without requiring exposureto radiation and/or the use of expensive medical procedures.

Embodiments of the present disclosure will be described more thoroughlyfrom now on with reference to the accompanying drawings. Like numeralsrepresent like elements throughout the several figures, and in whichexample embodiments are shown. However, embodiments of the claims can beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein. The examples set forth herein arenon-limiting examples and are merely examples, among other possibleexamples. Throughout this specification, plural instances (e.g., “602”)can implement components, operations, or structures (e.g., “602 a”)described as a single instance. Further, plural instances (e.g., “602”)refer collectively to a set of components, operations, or structures(e.g., “602 a”) described as a single instance. The description of asingle component (e.g., “602 a”) applies equally to a like-numberedcomponent (e.g., “602 b”) unless indicated otherwise. These and otheraspects, features, and implementations can be expressed as methods,apparatus, systems, components, program products, means or steps forperforming a function, and in other ways. These and other aspects,features, and implementations will become apparent from the followingdescriptions, including the claims.

Description of the Figures

FIG. 1 is a block diagram illustrating an example surgical system 100(“system 100”) in accordance with some embodiments of the presenttechnology. The system 100 includes various surgical and medicalequipment (e.g., a patient monitor 112) located within an operating room102 or a doctor's office 110, a console 108 for performing surgery orother patient care, and a database 106 for storing electronic healthrecords. The console 108 is the same as or similar to the console 420illustrated and described in more detail with reference to FIG. 4A. Thesystem 100 is implemented using the components of the example computersystem 300 illustrated and described in more detail with reference toFIG. 3 . Likewise, embodiments of the system 100 can include differentand/or additional components or can be connected in different ways.

The operating room 102 is a facility, e.g., within a hospital, wheresurgical operations are carried out in an aseptic environment. Propersurgical procedures require a sterile field. In some embodiments, thesterile field is maintained in the operating room 102 in a medical carefacility such as a hospital, the doctor's office 110, or outpatientsurgery center.

In some embodiments, the system 100 includes one or more medical orsurgical patient monitors 112. The monitors 112 can include a vitalsigns monitor (a medical diagnostic instrument), which can be aportable, battery powered, multi-parametric, vital signs monitoringdevice used for both ambulatory and transport applications as well asbedside monitoring. The vital signs monitor can be used with an isolateddata link to an interconnected portable computer or the console 108,allowing snapshot and trended data from the vital signs monitor to beprinted automatically at the console 108, and also allowing defaultconfiguration settings to be downloaded to the vital signs monitor. Thevital signs monitor is capable of use as a stand-alone unit as well aspart of a bi-directional wireless communications network that includesat least one remote monitoring station (e.g., the console 108). Thevital signs monitor can measure multiple physiological parameters of apatient wherein various sensor output signals are transmitted eitherwirelessly or by means of a wired connection to at least one remotesite, such as the console 108.

In some embodiments, the monitors 112 include a heart rate monitor,which is a sensor and/or a sensor system applied in the context ofmonitoring heart rates. The heart rate monitor measures, directly orindirectly, any physiological condition from which any relevant aspectof heart rate can be gleaned. For example, some embodiments of the heartrate monitor measure different or overlapping physiological conditionsto measure the same aspect of heart rate. Alternatively, someembodiments measure the same, different, or overlapping physiologicalconditions to measure different aspects of heart rate, e.g., number ofbeats, strength of beats, regularity of beats, beat anomalies, etc.

In some embodiments, the monitors 112 include a pulse oximeter or SpO2monitor, which is a plethysmograph or any instrument that measuresvariations in the size of an organ or body part of the patient on thebasis of the amount of blood passing through or present in the part. Thepulse oximeter is a type of plethysmograph that determines the oxygensaturation of the blood by indirectly measuring the oxygen saturation ofthe patient's blood (as opposed to measuring oxygen saturation directlythrough a blood sample) and changes in blood volume in the skin. Thepulse oximeter can include a light sensor that is placed at a site onthe patient, usually a fingertip, toe, forehead, or earlobe, or in thecase of a neonate, across a foot. Light, which can be produced by alight source integrated into the pulse oximeter, containing both red andinfrared wavelengths, is directed onto the skin of the patient, and thelight that passes through the skin is detected by the pulse oximeter.The intensity of light in each wavelength is measured by the pulseoximeter over time. The graph of light intensity versus time is referredto as the photoplethysmogram (PPG) or, more commonly, simply as the“pleth.” From the waveform of the PPG, it is possible to identify thepulse rate of the patient and when each individual pulse occurs. Inaddition, by comparing the intensities of two wavelengths when a pulseoccurs, it is possible to determine blood oxygen saturation ofhemoglobin in arterial blood. This relies on the observation that highlyoxygenated blood will relatively absorb more red light and less infraredlight than blood with a lower oxygen saturation.

In some embodiments, the monitors 112 include an end tidal CO2 monitoror capnography monitor used for measurement of the level of carbondioxide that is released at the end of an exhaled breath (referred to asend tidal carbon dioxide, ETCO2). An end tidal CO2 monitor orcapnography monitor is widely used in anesthesia and intensive care.ETCO2 can be calculated by plotting expiratory CO2 with time. Further,ETCO2 monitors are important for the measurement of applications such ascardiopulmonary resuscitation (CPR), airway assessment, proceduralsedation and analgesia, pulmonary diseases such as obstructive pulmonarydisease, pulmonary embolism, etc., heart failure, metabolic disorders,etc. The end tidal CO2 monitor can be configured as side stream(diverting) or mainstream (non-diverting). A diverting end tidal CO2monitor transports a portion of a patient's respired gases from thesampling site to the end tidal CO2 monitor while a non-diverting endtidal CO2 monitor does not transport gas away. Also, measurement by theend tidal CO2 monitor is based on the absorption of infrared light bycarbon dioxide where exhaled gas passes through a sampling chambercontaining an infrared light source and photodetector on both sides.Based on the amount of infrared light reaching the photodetector, theamount of carbon dioxide present in the gas can be determined.

In some embodiments, the monitors 112 include a blood pressure monitorthat measures blood pressure, particularly in arteries. The bloodpressure monitor uses a non-invasive technique (by external cuffapplication) or an invasive technique (by a cannula needle inserted inartery, used in the operating room 102) for measurement. Thenon-invasive method (referred to as a sphygmomanometer) works bymeasurement of force exerted against arterial walls during (i)ventricular systole (i.e., systolic blood pressure occurs when the heartbeats and pushes blood through the arteries) and (ii) ventriculardiastole (i.e., diastolic blood pressure occurs when the heart rests andis filling with blood) thereby measuring systole and diastole,respectively. The blood pressure monitor can be of three types:automatic/digital, manual (aneroid-dial), and manual (mercury-column).The sphygmomanometer can include a bladder, a cuff, a pressure meter, astethoscope, a valve, and a bulb. The cuff inflates until it fitstightly around the patient's arm, cutting off the blood flow, and thenthe valve opens to deflate it. The blood pressure monitor operates byinflating a cuff tightly around the arm; as the cuff reaches thesystolic pressure, blood begins to flow in the artery, creating avibration, which is detected by the blood pressure monitor, whichrecords the systolic pressure. The techniques used for measurement canbe auscultatory or oscillometric.

In some embodiments, the monitors 112 include a body temperaturemonitor. The body temperature monitor measures the temperatureinvasively or non-invasively by placement of a sensor into organs suchas bladder, rectum, esophagus, tympanum, etc., and mouth, armpit, etc.,respectively. The body temperature monitor is of two types: contact andnon-contact. Temperature can be measured in two forms: core temperatureand peripheral temperature. Temperature measurement can be done bythermocouples, resistive temperature devices (RTDs, thermistors),infrared radiators, bimetallic devices, liquid expansion devices,molecular change-of-state, and silicon diodes. A body temperaturemonitor commonly used for the measurement of temperature includes atemperature sensing element (e.g., temperature sensor) and a means forconverting to a numerical value.

In some embodiments, the monitors 112 measure respiration rate orbreathing rate—the rate at which breathing occurs—and which is measuredby the number of breaths the patient takes per minute. The rate ismeasured when a person is at rest and simply involves counting thenumber of breaths for one minute by counting how many times the chestrises. Normal respiration rates for an adult patient at rest are in therange: 12 to 16 breaths per minute. A variation can be an indication ofan abnormality/medical condition or the patient's demographicparameters. The monitors 112 can indicate hypoxia, a condition with lowlevels of oxygen in the cells, or hypercapnia, a condition in which highlevels of carbon dioxide are in the bloodstream. Pulmonary disorders,asthma, anxiety, pneumonia, heart diseases, dehydration, and drugoverdose are some abnormal conditions, which can cause a change to therespiration rate, thereby increasing or reducing the respiration ratefrom normal levels.

In some embodiments, the monitors 112 measure an electrocardiogram (EKGor ECG), a representation of the electrical activity of the heart(graphical trace of voltage versus time) by placement of electrodes onskin/body surface. The electrodes capture the electrical impulse, whichtravels through the heart causing systole and diastole or the pumping ofthe heart. This impulse provides information related to the normalfunctioning of the heart and the production of impulses. A change canoccur due to medical conditions such as arrhythmias (tachycardia, wherethe heart rate becomes faster, and bradycardia, where the heart ratebecomes slower), coronary heart disease, heart attacks, orcardiomyopathy. The instrument used for measurement of theelectrocardiogram is called an electrocardiograph, which measures theelectrical impulses by the placement of electrodes on the surface of thebody and represents the ECG by a PQRST waveform. A PQRST wave is readas: P wave, which represents the depolarization of the left and rightatrium and corresponds to atrial contraction; QRS complex, whichindicates ventricular depolarization and represents the electricalimpulse as it spreads through the ventricles; and T wave, whichindicates ventricular repolarization and follows the QRS complex.

In some embodiments, the monitors 112 perform neuromonitoring, alsocalled intraoperative neurophysiological monitoring (IONM). For example,the monitors 112 assess functions and changes in the brain, brainstem,spinal cord, cranial nerves, and peripheral nerves during a surgicalprocedure on these organs. Monitoring includes both continuousmonitoring of neural tissue as well as the localization of vital neuralstructures. IONM measures changes in these organs where the changes areindicative of irreversible damage or injuries in the organs, aiming atreducing the risk of neurological deficits after operations involvingthe nervous system. Monitoring is effective in localization ofanatomical structures, including peripheral nerves and the sensorimotorcortex, which helps in guiding the surgeon during dissection.Electrophysiological modalities employed in neuromonitoring are anextracellular single unit and local field recordings (LFP),somatosensory evoked potential (SSEP), transcranial electrical motorevoked potentials (TCeMEP), electromyography (EMG),electroencephalography (EEG), and auditory brainstem response (ABR). Theuse of neurophysiological monitoring during surgical procedures requiresanesthesia techniques to avoid interference and signal alteration due toanesthesia.

In some embodiments, the monitors 112 measure motor evoked potential(MEP), electrical signals that are recorded from descending motorpathways or muscles following stimulation of motor pathways within thebrain. MEP is determined by measurement of the action potential elicitedby non-invasive stimulation of the motor cortex through the scalp. MEPis for intraoperative monitoring and neurophysiological testing of themotor pathways specifically during spinal procedures. The technique ofmonitoring for measurement of MEP is defined based on parameters, suchas a site of stimulation (motor cortex or spinal cord), method ofstimulation (electrical potential or magnetic field), and site ofrecording (spinal cord or peripheral mixed nerve and muscle). The targetsite is stimulated by the use of electrical or magnetic means.

In some embodiments, the monitors 112 measure somatosensory evokedpotential (SSEP or SEP): the electrical signals generated by the brainand the spinal cord in response to sensory stimulus or touch. SSEP isused for intraoperative neurophysiological monitoring in spinalsurgeries. The measurements are reliable, which allows for continuousmonitoring during a surgical procedure. The sensor stimulus commonlygiven to the organs can be auditory, visual, or somatosensory SEPs andapplied on the skin, peripheral nerves of the upper limbs, lower limbs,or scalp. The stimulation technique can be mechanical, electrical(provides larger and more robust responses), or intraoperative spinalmonitoring modality.

In some embodiments, the monitors 112 provide electromyography (EMG):the evaluation and recording of electrical signals or electricalactivity of the skeletal muscles. An electromyography instrument,electromyograph, or electromyogram for the measurement of the EMGactivity records electrical activity produced by skeletal muscles andevaluates the functional integrity of individual nerves. The nervesmonitored by an EMG instrument can be intracranial, spinal, orperipheral nerves. The electrodes used for the acquisition of signalscan be invasive or non-invasive electrodes. The technique used formeasurement can be spontaneous or triggered. Spontaneous EMG refers tothe recording of myoelectric signals such as compression, stretching, orpulling of nerves during surgical manipulation Spontaneous EMG isrecorded by the insertion of a needle electrode. Triggered EMG refers tothe recording of myoelectric signals during stimulation of a target sitesuch as a pedicle screw with incremental current intensities.

In some embodiments, the monitors 112 provide electroencephalography(EEG), measuring the electrical signals in the brain. Brain cellscommunicate with each other through electrical impulses. EEG can be usedto help detect potential problems associated with this activity. Anelectroencephalograph is used for the measurement of EEG activity.Electrodes ranging from 8 to 16 pairs are attached to the scalp, whereeach pair of electrodes transmits a signal to one or more recordingchannels. EEG is a modality for intraoperative neurophysiologicalmonitoring and assessing cortical perfusion and oxygenation during avariety of vascular, cardiac, and neurosurgical procedures. The wavesproduced by EEG are alpha, beta, theta, and delta.

In some embodiments, the monitors 112 include sensors, such asmicrophones or optical sensors, that produce images or video capturedfrom at least one of multiple imaging devices, for example, camerasattached to manipulators or end effectors, cameras mounted to theceiling or other surface above the surgical theater, or cameras mountedon a tripod or other independent mounting device. In some embodiments,the cameras are body worn by a surgeon or other surgical staff, camerasare incorporated into a wearable device, such as an augmented realitydevice like Google Glass™, or cameras are integrated into an endoscopic,microscopic, or laparoscopic device. In some embodiments, a camera orother imaging device (e.g., ultrasound) present in the operating room102 is associated with one or more areas in the operating room 102. Thesensors can be associated with measuring a specific parameter of thepatient, such as respiratory rate, blood pressure, blood oxygen level,heart rate, etc.

In some embodiments, the system 100 includes a medical visualizationapparatus 114 used for visualization and analysis of objects (preferablythree-dimensional (3D) objects) in the operating room 102. The medicalvisualization apparatus 114 provides the selection of points atsurfaces, selection of a region of interest, or selection of objects.The medical visualization apparatus 114 can also be used for diagnosis,treatment planning, intraoperative support, documentation, oreducational purposes. The medical visualization apparatus 114 canfurther include microscopes, endoscopes/arthroscopes/laparoscopes, fiberoptics, surgical lights, high-definition monitors, operating roomcameras, etc. 3D visualization software provides visual representationsof scanned body parts via virtual models, offering significant depth andnuance to static two-dimensional medical images. The softwarefacilitates improved diagnoses, narrowed surgical operation learningcurves, reduced operational costs, and shortened image acquisitiontimes.

In some embodiments, the system 100 includes an instrument 118 such asan endoscope, arthroscope, or laparoscope for minimally invasive surgery(MIS), in which procedures are performed by cutting a minimal incisionin the body. An endoscope refers to an instrument used to visualize,diagnose, and treat problems inside hollow organs where the instrumentis inserted through natural body openings such as the mouth or anus. Anendoscope can perform a procedure as follows: a scope with a tiny cameraattached to a long, thin tube is inserted. The doctor moves it through abody passageway or opening to see inside an organ. It can be used fordiagnosis and surgery (such as for removing polyps from the colon). Anarthroscope refers to an instrument used to visualize, diagnose, andtreat problems inside a joint by a TV camera inserted through smallportals/incisions and to perform procedures on cartilage, ligaments,tendons, etc. An arthroscope can perform the procedure as follows: asurgeon makes a small incision in a patient's skin and inserts apencil-sized instrument with a small lens and lighting system to magnifythe target site (joint) and viewing of the interior of the joint bymeans of a miniature TV camera and then performs the procedure. Alaparoscope refers to an instrument used to visualize, diagnose, andtreat problems inside soft organs like the abdomen and pelvis by a TVcamera inserted through small portals/incisions and to performprocedures.

In some embodiments, the system 100 includes fiber optics 120, whichrefer to flexible, transparent fiber made by drawing glass (silica) orplastic to a diameter slightly thicker than that of a human hair. Fiberoptics 120 are arranged in bundles called optical cables and used totransmit light signals across long distances. Fiber optics 120 are usedmost often as a means to transmit light between the two ends of thefiber and find wide usage in the medical field. Traditional surgeryrequires sizable and invasive incisions to expose internal organs andoperate on affected areas, but with fiber optics 120 much smallersurgical incisions can be performed. Fiber optics 120 contain componentssuch as a core, cladding, and buffer coating. Fiber optics 120 can beinserted in hypodermic needles and catheters, endoscopes, operationtheater tools, ophthalmological tools, and dentistry tools. Fiber opticsensors include a light source, optical fiber, external transducer, andphotodetector. Fiber optic sensors can be intrinsic or extrinsic. Fiberoptic sensors can be categorized into four types: physical, imaging,chemical, and biological.

In some embodiments, the system 100 includes surgical lights 122(referred to as operating lights) that perform illumination of a localarea or cavity of the patient. Surgical lights 122 play an importantrole in illumination before, during, and after a medical procedure.Surgical lights 122 can be categorized by lamp type as conventional(incandescent) and LED (light-emitting diode). Surgical lights 122 canbe categorized by mounting configuration as ceiling-mounted,wall-mounted, or floor stand. Surgical lights 122 can be categorized bytype as tungsten, quartz, xenon halogens, and/or LEDs. Surgical lights122 include sterilizable handles, which allow the surgeon to adjustlight positions. Some important factors affecting surgical lights 122can be illumination, shadow management (cast shadows and contourshadows), the volume of light, heat management, or fail-safe surgicallighting.

In some embodiments, the system 100 includes a surgical tower 128, e.g.,used in conjunction with the robotic surgical system 160 disclosedherein, for MIS. The surgical tower 128 includes instruments used forperforming MIS or surgery, which is performed by creating smallincisions in the body. The instruments are also referred to as minimallyinvasive devices or minimally invasive access devices. The procedure ofperforming MIS can also be referred to as a minimally invasiveprocedure. MIS is a safer, less invasive, and more precise surgicalprocedure. Some medical procedures where the surgical tower 128 isuseful and widely used are procedures for lung, gynecological, head andneck, heart, and urological conditions. MIS can be robotic ornon-robotic/endoscopic. MIS can include endoscopic, laparoscopic,arthroscopic, natural orifice intraluminal, and natural orificetransluminal procedures. A surgical tower access device can also bedesigned as an outer sleeve and an inner sleeve that telescopingly orslidably engage with one another. When a telescope is used to operate onthe abdomen, the procedure is called laparoscopy. The surgical tower 128typically includes access to a variety of surgical tools, such as forelectrocautery, radiofrequency, lasers, sensors, etc.

In some embodiments, radiofrequency (RF) is used in association with MISdevices. The RF can be used for the treatment of skin by delivering itto the skin through a minimally invasive surgical tool (e.g., fineneedles), which does not require skin excision. The RF can be used forreal-time tracking of MIS devices such as laparoscopic instruments. TheRF can provide radiofrequency ablation to a patient suffering fromatrial fibrillation through smaller incisions made between the ribs. TheRF can be used to perform an endoscopic surgery on the body such as thespine by delivery of RF energy.

In some embodiments, the system 100 includes an instrument 130 toperform electrocautery for burning a part of the body to remove or closeoff a part of it. Various physiological conditions or surgicalprocedures require the removal of body tissues and organs, a consequenceof which is bleeding. In order to achieve hemostasis and for removingand sealing all blood vessels that are supplied to an organ aftersurgical incision, the electrocautery instrument 130 can be used. Forexample, after removing part of the liver for removal of a tumor, etc.,blood vessels in the liver must be sealed individually. Theelectrocautery instrument 130 can be used for sealing living tissue suchas arteries, veins, lymph nodes, nerves, fats, ligaments, and other softtissue structures. The electrocautery instrument 130 can be used inapplications such as surgery, tumor removal, nasal treatment, or wartremoval. Electrocautery can operate in two modes, monopolar or bipolar.The electrocautery instrument can 130 consist of a generator, ahandpiece, and one or more electrodes.

In some embodiments, the system 100 includes a laser 132 used inassociation with MIS devices. The laser 132 can be used in MIS with anendoscope. The laser 132 is attached to the distal end of the endoscopeand steered at high speed by producing higher incision quality than withexisting surgical tools thereby minimizing damage to surrounding tissue.The laser 132 can be used to perform MIS using a laparoscope in thelower and upper gastrointestinal tract, eye, nose, and throat. The laser132 is used in MIS to ablate soft tissues, such as a herniated spinaldisc bulge.

In some embodiments, sensors 134 are used in association with MISdevices and the robotic surgical system 160 described herein. Thesensors 134 can be used in MIS for tactile sensing of surgicaltool-tissue interaction forces. During MIS, the field of view andworkspace of surgical tools are compromised due to the indirect accessto the anatomy and lack of surgeon's hand-eye coordination. The sensors134 provide a tactile sensation to the surgeon by providing informationregarding shape, stiffness, and texture of organ or tissue (differentcharacteristics) to the surgeon's hands through a sense of touch. Thisdetects a tumor through palpation, which exhibits a “tougher” feel thanthat of healthy soft tissue, pulse felt from blood vessels, and abnormallesions. The sensors 134 can output shape, size, pressure, softness,composition, temperature, vibration, shear, and normal forces. Thesensors 134 can be electrical or optical, consisting of capacitive,inductive, piezoelectric, piezoresistive, magnetic, and auditory. Thesensors 134 can be used in robotic or laparoscopic surgery, palpation,biopsy, heart ablation, and valvuloplasty.

In some embodiments, the system 100 includes an imaging system 136(instruments are used for the creation of images and visualization ofthe interior of a human body for diagnostic and treatment purposes). Theimaging system 136 is used in different medical settings and can help inthe screening of health conditions, diagnosing causes of symptoms, ormonitoring of health conditions. The imaging system 136 can includevarious imaging techniques such as X-ray, fluoroscopy, magneticresonance imaging (MRI), ultrasound, endoscopy, elastography, tactileimaging, thermography, medical photography, and nuclear medicine, e.g.,positron emission tomography (PET). Some factors which can drive themarket are cost and clinical advantages of medical imaging modalities, arising share of ageing populations, increasing prevalence ofcardiovascular or lifestyle diseases, and increasing demand fromemerging economies.

In some embodiments, the imaging system 136 includes X-ray medicalimaging instruments that use X-ray radiation (i.e., X-ray range in theelectromagnetic radiation spectrum) for the creation of images of theinterior of the human body for diagnostic and treatment purposes. AnX-ray instrument is also referred to as an X-ray generator. It is anon-invasive instrument based on different absorption of X-rays bytissues based on their radiological density (radiological density isdifferent for bones and soft tissues). For the creation of an image bythe X-ray instrument, X-rays produced by an X-ray tube are passedthrough a patient positioned to the detector. As the X-rays pass throughthe body, images appear in shades of black and white, depending on thetype and densities of tissue the X-rays pass through. Some of theapplications where X-rays are used can be bone fractures, infections,calcification, tumors, arthritis, blood vessel blockages, digestiveproblems, or heart problems. The X-ray instrument can consist ofcomponents such as an X-ray tube, operating console, collimator, grid,detector, radiographic film, etc.

In some embodiments, the imaging system 136 includes Mill medicalimaging instruments that use powerful magnets for the creation of imagesof the interior of the human body for diagnostic and treatment purposes.Some of the applications where MM can be used can be brain/spinal cordanomalies, tumors in the body, breast cancer screening, joint injuries,uterine/pelvic pain detection, or heart problems. For the creation ofthe image by an Mill instrument, magnetic resonance is produced bypowerful magnets, which produce a strong magnetic field that forcesprotons in the body to align with that field. When a radiofrequencycurrent is then pulsed through the patient, the protons are stimulated,and spin out of equilibrium, straining against the pull of the magneticfield. Turning off the radiofrequency field allows detection of energyreleased by realignment of protons with the magnetic field by MRIsensors. The time taken by the protons for realignment with the magneticfield and energy release is dependent on environmental factors and thechemical nature of the molecules. MM is more widely suitable for imagingof non-bony parts or soft tissues of the body. MM can be less harmful asit does not use damaging ionizing radiation as in the X-ray instrument.MM instruments can consist of magnets, gradients, radiofrequencysystems, or computer control systems. Some areas where imaging by MMshould be prohibited can be people with implants.

In some embodiments, the imaging system 136 uses computed tomographyimaging (CT) that uses an X-ray radiation (i.e., X-ray range in theelectromagnetic radiation spectrum) for the creation of cross-sectionalimages of the interior of the human body. CT refers to a computerizedX-ray imaging procedure in which a narrow beam of X-rays is aimed at apatient and quickly rotated around the body, producing signals that areprocessed by the machine's computer to generate cross-sectionalimages—or “slices”—of the body. A CT instrument is different from anX-ray instrument as it creates 3-dimensional (3D) cross-sectional imagesof the body while the X-ray instrument creates 2-dimensional (2D) imagesof the body; the 3D cross-sectional images are created by taking imagesfrom different angles, which is done by taking a series of tomographicimages from different angles. The diverse images are collected by acomputer and digitally stacked to form a 3D image of the patient. Forcreation of images by the CT instrument, a CT scanner uses a motorizedX-ray source that rotates around the circular opening of a donut-shapedstructure called a gantry while the X-ray tube rotates around thepatient shooting narrow beams of X-rays through the body. Some of theapplications where CT can be used can be blood clots; bone fractures,including subtle fractures not visible on X-ray; or organ injuries.

In some embodiments, the imaging system 136 includes ultrasound imaging,also referred to as sonography or ultrasonography, that uses ultrasoundor sound waves (also referred to as acoustic waves) for the creation ofcross-sectional images of the interior of the human body. Ultrasoundwaves in the imaging system 136 can be produced by a piezoelectrictransducer, which produces sound waves and sends them into the body. Thesound waves that are reflected are converted into electrical signals,which are sent to an ultrasound scanner. Ultrasound instruments can beused for diagnostic and functional imaging or for therapeutic orinterventional procedures. Some of the applications where ultrasound canbe used are diagnosis/treatment/guidance during medical procedures(e.g., biopsies, internal organs such as liver/kidneys/pancreas, fetalmonitoring, etc.), in soft tissues, muscles, blood vessels, tendons, orjoints. Ultrasound can be used for internal imaging (where thetransducer is placed in organs, e.g., vagina) and external imaging(where the transducer is placed on the chest for heart monitoring or theabdomen for fetal monitoring). An ultrasound machine can consist of amonitor, keyboard, processor, data storage, probe, and transducer.

In some embodiments, the system 100 includes a stereotactic navigationsystem 138 that uses patient imaging (e.g., CT, MRI) to guide surgeonsin the placement of specialized surgical instruments and implants. Thepatient images are taken to guide the physician before or during themedical procedure. The stereotactic navigation system 138 includes acamera having infrared sensors to determine the location of the tip ofthe probe being used in the surgical procedure. This information is sentin real-time so that the surgeons have a clear image of the preciselocation where they are working in the body. The stereotactic navigationsystem 138 can be framed (requires attachment of a frame to thepatient's head using screws or pins) or frameless (does not require theplacement of a frame on the patient's anatomy). The stereotacticnavigation system 138 can be used for diagnostic biopsies, tumorresection, bone preparation/implant placement, placement of electrodes,otolaryngologic procedures, or neurosurgical procedures.

In some embodiments, the system 100 includes an anesthesiology machine140 that is used to generate and mix medical gases, such as oxygen orair, and anesthetic agents to induce and maintain anesthesia inpatients. The anesthesiology machine 140 delivers oxygen and anestheticgas to the patient and filters out expiratory carbon dioxide. Theanesthesiology machine 140 can perform functions such as providingoxygen (02), accurately mixing anesthetic gases and vapors, enablingpatient ventilation, and minimizing anesthesia-related risks to patientsand staff. The anesthesiology machine 140 can include the followingessential components: a source of O2, O2 flowmeter, vaporizer(anesthetics include isoflurane, halothane, enflurane, desflurane,sevoflurane, and methoxyflurane), patient breathing circuit (tubing,connectors, and valves), and scavenging system (removes any excessanesthetics gases). The anesthesiology machine 140 can be divided intothree parts: the high pressure system, the intermediate pressure system,and the low pressure system. The process of anesthesia starts withoxygen flow from a pipeline or cylinder through the flowmeter; the O2flows through the vaporizer and picks up the anesthetic vapors; theO2-anesthetic mix then flows through the breathing circuit and into thepatient's lungs, usually by spontaneous ventilation or normalrespiration.

In some embodiments, the system 100 includes a surgical bed 142 equippedwith mechanisms that can elevate or lower the entire bed platform; flex,or extend individual components of the platform; or raise or lower thehead or the feet of the patient independently. The surgical bed 142 canbe an operation bed, cardiac bed, amputation bed, or fracture bed. Someessential components of the surgical bed 142 can be a bed sheet, woolenblanket, bath towel, and bed block. The surgical bed 142 can also bereferred to as a post-operative bed, which refers to a special type ofbed made for the patient who is coming from the operation theater orfrom another procedure that requires anesthesia. The surgical bed 142 isdesigned in a manner that makes it easier to transfer an unconscious orweak patient from a stretcher/wheelchair to the bed. The surgical bed142 should protect bed linen from vomiting, bleeding, drainage, anddischarge; provide warmth and comfort to the patient to prevent shock;provide necessary positions, which are suitable for operation; protectpatient from being chilled; and be prepared to meet any emergency.

In some embodiments, the system 100 includes a Jackson frame 144 (orJackson table), which refers to a frame or table that is designed foruse in spinal surgeries and can be used in a variety of spinalprocedures in supine, prone, or lateral positions in a safe manner. Twopeculiar features of the Jackson table 144 are the absence of centraltable support and an ability to rotate the table through 180 degrees.The Jackson table 144 is supported at both ends, which keeps the wholeof the table free. This allows the visualization of a patient's trunkand major parts of extremities as well. The Jackson frame 144 allows thepatient to be slid from the cart onto the table in the supine positionwith appropriate padding placed. The patient is then strapped securelyon the Jackson table 144.

In some embodiments, the system 100 includes a disposable air warmer 146(sometimes referred to as a Bair™ or Bair Hugger™). The disposable airwarmer 146 is a convective temperature management system used in ahospital or surgery center to maintain a patient's core bodytemperature. The disposable air warmer 146 includes a reusable warmingunit and a single-use disposable warming blanket for use during surgery.It can also be used before and after surgery. The disposable air warmer146 uses convective warming consisting of two components: a warming unitand a disposable blanket. The disposable air warmer 146 filters air andthen forces warm air through disposable blankets, which cover thepatient. The blanket can be designed to use pressure points on thepatient's body to prevent heat from reaching areas at risk for pressuresores or burns. The blanket can also include drainage holes where fluidpasses through the surface of the blanket to linen underneath, whichwill reduce the risk of skin softening and reduce the risk of unintendedcooling because of heat loss from evaporation.

In some embodiments, the system 100 includes a sequential compressiondevice (SCD) 148 used to help prevent blood clots in the deep veins oflegs. The sequential compression device 148 uses cuffs around the legsthat fill with air and squeeze the legs. This increases blood flowthrough the veins of the legs and helps prevent blood clots. A deep veinthrombosis (DVT) is a blood clot that forms in a vein deep inside thebody. Some of the risks of using the SCD 148 can be discomfort, warmth,sweating beneath the cuff, skin breakdown, nerve damage, or pressureinjury.

In some embodiments, the system 100 includes a bed position controller150, which refers to an instrument for controlling the position of thepatient bed. Positioning a patient in bed is important for maintainingalignment and for preventing bedsores (pressure ulcers), foot drop, andcontractures. Proper positioning is also vital for providing comfort forpatients who are bedridden or have decreased mobility related to amedical condition or treatment. When positioning a patient in bed,supportive devices such as pillows, rolls, and blankets, along withrepositioning, can aid in providing comfort and safety. The patient canbe in the following positions in a bed: supine position, prone position,lateral position, Sims' position, Fowler's position, semi-Fowler'sposition, orthopedic or tripod position, or Trendelenburg position.

In some embodiments, the system 100 includes environmental controls 152.The environmental controls 152 can be operating room environmentalcontrols for control or maintenance of the environment in the operatingroom 102 where procedures are performed to minimize the risk of airborneinfection and to provide a conducive environment for everyone in theoperating room 102 (e.g., surgeon, anesthesiologist, nurses, andpatient). Some factors that can contribute to poor quality in theenvironment of the operating room 102 are temperature, ventilation, andhumidity, and those conditions can lead to profound effects on thehealth and work productivity of people in the operating room 102. As anexample: surgeons prefer a cool, dry climate since they work underbright, hot lights; anesthesia personnel prefer a warmer, less breezyclimate; patient condition demands a relatively warm, humid, and quietenvironment. The operating room environmental controls can control theenvironment by taking care of the following factors: environmentalhumidity, infection control, or odor control. Humidity control can beperformed by controlling the temperature of anesthesia gases; infectioncan be controlled by the use of filters to purify the air.

In some embodiments, the environmental controls 152 include a heating,ventilation, and air conditioning (HVAC) system for regulating theenvironment of indoor settings by moving air between indoor and outdoorareas, along with heating and cooling. HVAC can use a differentcombination of systems, machines, and technologies to improve comfort.HVAC can be necessary to maintain the environment of the operating room102. The operating room 102 can be a traditional operating room (whichcan have a large diffuser array directly above the operating table) or ahybrid operating room (which can have monitors and imaging equipment 136that consume valuable ceiling space and complicate the design process).HVAC can include three main units, for example, a heating unit (e.g.,furnace or boiler), a ventilation unit (natural or forced), and an airconditioning unit (which can remove existing heat). HVAC can be made ofcomponents such as air returns, filters, exhaust outlets, ducts,electrical elements, outdoor units, compressors, coils, and blowers. TheHVAC system can use central heating and AC systems that use a singleblower to circulate air via internal ducts.

In some embodiments, the environmental controls 152 include an airpurification system for removing contaminants from the air in theoperating room 102 to improve indoor air quality. Air purification canbe important in the operating room 102 as surgical site infection can bea reason for high mortality and morbidity. The air purification systemcan deliver clean, filtered, contaminant-free air over the surgical bed142 using a diffuser, airflow, etc., to remove all infectious particlesdown and away from the patient. The air purification system can be anair curtain, multi-diffuser array, or single large diffuser (based onlaminar diffuser flow) or High-Efficiency Particulate Air filter (HEPAfilter). A HEPA filter protects a patient from infection andcontamination using a filter, which is mounted at the terminal of theduct. A HEPA filter can be mounted on the ceiling and deliver clean,filtered air in a flow to the operating room 102 that provides asweeping effect that pushes contaminants out via the return grilles thatare usually mounted on the lower wall.

In some embodiments, the system 100 includes one or more medical orsurgical tools 154. The surgical tools 154 can include orthopedic tools(also referred to as orthopedic instruments) used for treatment andprevention of deformities and injuries of the musculoskeletal system orskeleton, articulations, and locomotive system (i.e., set formed byskeleton, muscles attached to it, and the part of the nervous systemthat controls the muscles). A major percentage of orthopedic tools aremade of plastic. The orthopedic tools can be divided into the followingspecialties: hand and wrist, foot and ankle, shoulder, and elbow,arthroscopic, hip, and knee. The orthopedic tools can be fixation tools,relieving tools, corrective tools, or compression-distraction tools. Afixation tool refers to a tool designed to restrict movements partiallyor completely in a joint, e.g., hinged splints (for preserving a certainrange of movement in a joint) or rigid splints. A relieving tool refersto a tool designed to relieve pressure on an ailing part by transferringsupport to healthy parts of an extremity, e.g., Thomas splint and theVoskoboinikova apparatus. A corrective tool refers to a surgical tooldesigned to gradually correct a deformity, e.g., corsets, splints,orthopedic footwear, insoles, and other devices to correct abnormalpositions of the foot. A compression-distraction tool refers to asurgical tool designed to correct acquired or congenital deformities ofthe extremities, e.g., curvature, shortening, and pseudarthrosis such asGudushauri. A fixation tool can be an internal fixation tool (e.g.,screws, plates) or external fixation tools used to correct a radius ortibia fracture. The orthopedic tools can be bone-holding forceps, drillbits, nail pins, hammers, staples, etc.

In some embodiments, the surgical tools 154 include a drill for makingholes in bones for insertion of implants like nails, plates, screws, andwires. The drill tool functions by drilling cylindrical tunnels intobone. Drills can be used in orthopedics for performing medicalprocedures. If the drill does not stop immediately when used, the use ofthe drill on bones can have some risks, such as harm caused to bone,muscle, nerves, and venous tissues, which are wrapped by surroundingtissue. Drills vary widely in speed, power, and size. Drills can bepowered as electrical, pneumatic, or battery. Drills generally can workon speeds below 1000 rpm in orthopedic settings. Temperature control ofdrills is an important aspect in the functioning of the drill and isdependent on parameters such as rotation speed, torque, orthotropicsite, sharpness of the cutting edges, irrigation, and cooling systems.The drill can include a physical drill, power cord, electronicallymotorized bone drill, or rotating bone shearing incision work unit.

In some embodiments, the surgical tools 154 include a scalpel forslicing, cutting, or osteotomy of bone during orthopedic procedure. Thescalpel can be designed to provide clean cuts through osseous structureswith minimal loss of viable bone while sparing adjacent elastic softtissues largely unaffected while performing a slicing procedure. This issuited for spine applications where bone must be cut adjacent to thedura and neural structures. The scalpel does not rotate but performscutting by an ultrasonically oscillating or forward/backward movingmetal tip. Scalpels can prevent injuries caused by a drill in a spinalsurgery such as complications such as nerve thermal injury, graspingsoft tissue, tearing dura mater, and mechanical injury.

In some embodiments, stitches (also referred to as sutures) or asterile, surgical thread is used to repair cuts or lacerations and isused to close incisions or hold body tissues together after a surgery oran injury. Stitches can involve the use of a needle along with anattached thread. Stitches can be either absorbable (the stitchesautomatically break down harmlessly in the body over time withoutintervention) or non-absorbable (the stitches do not automatically breakdown over time and must be manually removed if not left indefinitely).Stitches can be based on material monofilament, multifilament, and barb.Stitches can be classified based on size. Stitches can be based onsynthetic or natural material. Stitches can be coated or un-coated.

In some embodiments, the surgical tools 154 include a stapler used forfragment fixation when inter-fragmental screw fixation is not easy. Whenthere is vast damage and a bone is broken into fragments, staples can beused between these fragments for internal fixation and bonereconstruction. For example, they can be used around joints in ankle andfoot surgeries, in cases of soft tissue damage, or to attach tendons orligaments to the bone for reconstruction surgery. Staplers can be madeof surgical grade stainless steel or titanium, and they are thicker,stronger, and larger.

In some embodiments, other medical or surgical equipment, such as a setof articles, surgical tools, or objects, is used to implement or achievean operation or activity. A medical equipment refers to an article,instrument, apparatus, or machine used for diagnosis, prevention, ortreatment of a medical condition or disease, or to the detection,measurement, restoration, correction, or modification ofstructure/function of the body for some health purpose. The medicalequipment can perform functions invasively or non-invasively. In someembodiments, the medical equipment includes components such as asensor/transducer, a signal conditioner, a display, or a data storageunit, etc. In some embodiments, the medical equipment includes a sensorto receive a signal from instruments measuring a patient's body, atransducer for converting one form of energy to electrical energy, asignal conditioner such as an amplifier, filter, etc., to convert theoutput from the transducer into an electrical value, a display toprovide a visual representation of the measured parameter or quantity,or a storage system to store data, which can be used for futurereference. A medical equipment can perform diagnosis or provide therapy;for example, the equipment delivers air into the lungs of a patient whois physically unable to breathe, or breathes insufficiently, and movesit out of the lungs.

In some embodiments, the system includes a machine 156 to aid inbreathing. The machine 156 can be a ventilator (also referred to as arespirator) that provides a patient with oxygen when they are unable tobreathe on their own. A ventilator is required when a person is not ableto breathe on their own. A ventilator can perform a function of gentlypushing air into the lungs and allow it to come back out. The ventilatorfunctions by delivery of positive pressure to force air into the lungs,while usual breathing uses negative pressure by the opening of themouth, and air flows in. The ventilator can be required during surgeryor after surgery. The ventilator can be required in case of respiratoryfailure due to acute respiratory distress syndrome, head injury, asthma,lung diseases, drug overdose, neonatal respiratory distress syndrome,pneumonia, sepsis, spinal cord injury, cardiac arrest, etc., or duringsurgery. The ventilator can be used with a face mask (non-invasiveventilation, where the ventilation is required for a shorter duration oftime) or with a breathing tube also referred to as an endotracheal tube(invasive ventilation, where the ventilation is required for a longerduration of time). Ventilator use can have some risks such asinfections, fluid build-up, muscle weakness, lung damage, etc. Theventilator can be operated in various modes, such as assist-controlventilation (ACV), synchronized intermittent-mandatory ventilation(SIMV), pressure-controlled ventilation (PCV), pressure supportventilation (PSV), pressure-controlled inverse ratio ventilation(PCIRV), airway pressure release ventilation (APRV), etc. The ventilatorcan include a gas delivery system, power source, control system, safetyfeature, gas filter, and monitor.

In some embodiments, the machine 156 is a continuous positive airwaypressure (CPAP) used for the treatment of sleep apnea disorder in apatient. Sleep apnea refers to a disorder in which breathing repeatedlystops and starts while a patient is sleeping, often becausethroat/airways briefly collapse or something temporarily blocks them.Sleep apnea can lead to serious health problems, such as high bloodpressure and heart trouble. A CPAP instrument helps the patient withsleep apnea to breathe more easily during sleep by sending a steady flowof oxygen into the nose and mouth during sleep, which keeps the airwaysopen and helps the patient to breathe normally. The CPAP machine canwork by a compressor/motor, which generates a continuous stream ofpressurized air that travels through an air filter into a flexible tube.The tube delivers purified air into a mask sealed around the nose/mouthof the patient. The airstream from the instrument pushes against anyblockages, opening the airways so lungs receive plenty of oxygen, andbreathing does not stop as nothing obstructs oxygen. This helps thepatient to not wake up to resume breathing. CPAP can have a nasal pillowmask, nasal mask, or full mask. CPAP instrument can include a motor, acushioned mask, a tube that connects the motor to the mask, a headgearframe, and adjustable straps. The essential components can be a motor, acushioned mask, and a tube that connects the motor to the mask.

In some embodiments, the system 100 includes surgical supplies,consumables 158, or necessary supplies for the system 100 to providecare within the hospital or surgical environment. The consumables 158can include gloves, gowns, masks, syringes, needles, sutures, staples,tubing, catheters, or adhesives for wound dressing, in addition to othersurgical tools needed by doctors and nurses to provide care. Dependingon the device, mechanical testing can be carried out in tensile,compression, or flexure; in dynamic or fatigue; via impact; or with theapplication of torsion. The consumables 158 can be disposable (e.g.,time-saving, have no risk of healthcare-associated infections, andcost-efficient) or sterilizable (to avoid cross-contamination or risk ofsurgical site infections).

In some embodiments, the system 100 includes a robotic surgical system160 (sometimes referred to as a medical robotic system or a roboticsystem) that provides intelligent services and information to theoperating room 102 and the console 108 by interacting with theenvironment, including human beings, via the use of various sensors,actuators, and human interfaces. The robotic surgical system 160 can beemployed for automating processes in a wide range of applications,ranging from industrial (manufacturing), domestic, medical, service,military, entertainment, space, etc. The medical robotic system marketis segmented by product type into surgical robotic systems,rehabilitative robotic systems, non-invasive radiosurgery robots, andhospital and pharmacy robotic systems. Robotic surgeries are performedusing tele-manipulators (e.g., input devices 166 at the console 108),which use the surgeon's actions on one side to control one or more“effectors” on the other side. The medical robotic system 160 providesprecision and can be used for remotely controlled, minimally invasiveprocedures. The robotic surgical system 160 includes computer-controlledelectromechanical devices that work in response to controls (e.g., inputdevices 166 at the console 108) manipulated by the surgeons.

In some embodiments, the system 100 includes equipment tracking systems162, such as RFID, which is used to tag an instrument with an electronictag and tracks it using the tag. Typically, this could involve acentralized platform that provides details such as location, owner,contract, and maintenance history for all equipment in real-time. Avariety of techniques can be used to track physical assets, includingRFID, global positioning system (GPS), Bluetooth low energy (BLE),barcodes, near-field communication (NFC), Wi-Fi, etc. The equipmenttracking system 162 includes hardware components, such as RFID tags, GPStrackers, barcodes, and QR codes. The hardware component is placed onthe asset, and it communicates with the software (directly or via ascanner), providing the software with data about the asset's locationand properties. In some embodiments, the equipment tracking system 162uses electromagnetic fields to transmit data from an RFID tag to areader. Reading of RFID tags can be done by portable or mounted RFIDreaders. The read range for RFID varies with the frequency used.Managing and locating important assets is a key challenge for trackingmedical equipment. Time spent searching for critical equipment can leadto expensive delays or downtime, missed deadlines and customercommitments, and wasted labor. The problem has previously been solved byusing barcode labels or manual serial numbers and spreadsheets; however,these require manual labor. The RFID tag can be passive (smaller andless expensive, read ranges are shorter, have no power of their own, andare powered by the radio frequency energy transmitted from RFIDreaders/antennas) or active (larger and more expensive, read ranges arelonger, have a built-in power source and transmitter of their own).

In some embodiments, the system 100 includes medical equipment,computers, software, etc., located in the doctor's office 110 that iscommunicably coupled to the operating room 102 over the network 104. Forexample, the medical equipment in the doctor's office 110 can include amicroscope 116 used for viewing samples and objects that cannot be seenwith an unaided eye. The microscope 116 can have components such aseyepieces, objective lenses, adjustment knobs, a stage, an illuminator,a condenser, or a diaphragm. The microscope 116 works by manipulatinghow light enters the eye using a convex lens, where both sides of thelens are curved outwards. When light reflects off of an object beingviewed under the microscope 116 and passes through the lens, it bendstoward the eye. This makes the object look bigger than it is. Themicroscope 116 can be compound (light-illuminated and the image seenwith the microscope 116 is two-dimensional), dissection or stereoscope(light-illuminated and the image seen with the microscope 116 isthree-dimensional), confocal (laser-illuminated and the image seen withthe microscope 116 is on a digital computer screen), scanning electron(SEM) (electron-illuminated and the image seen with the microscope 116is in black and white), or transmission electron microscope (TEM)(electron-illuminated and the image seen with the microscope 116 is thehigh magnification and high resolution).

The system 100 includes an electronic health records (EHR) database 106that contains patient records. The EHR are a digital version ofpatients' paper charts. The EHR database 106 can contain moreinformation than a traditional patient chart, including, but not limitedto, a patients' medical history, diagnoses, medications, treatmentplans, allergies, diagnostic imaging, lab results, etc. In someembodiments, the steps for each procedure disclosed herein are stored inthe EHR database 106. Electronic health records can also include datacollected from the monitors 112 from historical procedures. The EHRdatabase 106 is implemented using components of the example computersystem 300 illustrated and described in more detail with reference toFIG. 3 .

In some embodiments, the EHR database 106 includes a digital record ofpatients' health information, collected, and stored systematically overtime. The EHR database 106 can include demographics, medical history,history of present illness (HPI), progress notes, problems, medications,vital signs, immunizations, laboratory data, or radiology reports.Software (in memory 164) operating on the console 108 or implemented onthe example computer system 300 (e.g., the instructions 304, 308illustrated and described in more detail with reference to FIG. 3 ) areused to capture, store, and share patient data in a structured way. TheEHR database 106 can be created and managed by authorized providers andcan make health information accessible to authorized providers acrosspractices and health organizations, such as laboratories, specialists,medical imaging facilities, pharmacies, emergency facilities, etc. Thetimely availability of EHR data enables healthcare providers to makemore accurate decisions and provide better care to the patients byeffective diagnosis and reduced medical errors. Besides providingopportunities to enhance patient care, the EHR database 106 can also beused to facilitate clinical research by combining patients' demographicsinto a large pool. For example, the EHR database 106 can support a widerange of epidemiological research on the natural history of disease,drug utilization, and safety, as well as health services research.

The console 108 is a computer device, such as a server, computer,tablet, smartphone, smart speaker, etc., implemented using components ofthe example computer system 300 illustrated and described in more detailwith reference to FIG. 3 . In some embodiments, the steps for eachprocedure disclosed herein are stored in memory 164 on the console 108for execution.

In some embodiments, the operating room 102 or the console 108 includeshigh-definition monitors 124, which refer to displays in which a clearerpicture is possible than with low-definition, low-resolution screens.The high-definition monitors 124 have a higher density of pixels perinch than past standard TV screens. Resolution for the high-definitionmonitors 124 can be 1280×720 pixels or more (e.g., Full HD, 1920×1080;Quad HD, 2560×1440; 4K, 3840×2160; 8K, 7680×4320 pixels). Thehigh-definition monitor 124 can operate in progressive or interlacedscanning mode. High-definition monitors used in medical applications canoffer improved visibility; allow for precise and safe surgery with richcolor reproduction; provide suitable colors for each clinicaldiscipline; provide better visibility, operability with a large screenand electronic zoom, higher image quality in low light conditions,better visualization of blood vessels and lesions, and high contrast athigh spatial frequencies; be twice as sensitive as conventional sensors;and make it easier to determine tissue boundaries (fat, nerves, vessels,etc.).

In some embodiments, the console 108 includes an input interface or oneor more input devices 166. The input devices 166 can include a keyboard,a mouse, a joystick, any hand-held controller, or a hand-controlledmanipulator, e.g., a tele-manipulator used to perform robotic surgery.

In some embodiments, the console 108, the equipment in the doctor'soffice 110, and the EHR database 106 are communicatively coupled to theequipment in the operating room 102 by a direct connection, such asethernet, or wirelessly by the cloud over the network 104. The network104 is the same as or similar to the network 314 illustrated anddescribed in more detail with reference to FIG. 3 . For example, theconsole 108 can communicate with the robotic surgical system 160 usingthe network adapter 312 illustrated and described in more detail withreference to FIG. 3 .

FIG. 2 is a block diagram illustrating an example machine learning (ML)system 200, in accordance with one or more embodiments. The ML system200 is implemented using components of the example computer system 300illustrated and described in more detail with reference to FIG. 3 . Forexample, the ML system 200 can be implemented on the console 108 usinginstructions programmed in the memory 164 illustrated and described inmore detail with reference to FIG. 1 . Likewise, embodiments of the MLsystem 200 can include different and/or additional components or beconnected in different ways. The ML system 200 is sometimes referred toas a ML module.

The ML system 200 includes a feature extraction module 208 implementedusing components of the example computer system 300 illustrated anddescribed in more detail with reference to FIG. 3 . In some embodiments,the feature extraction module 208 extracts a feature vector 212 frominput data 204. For example, the input data 204 can include one or morephysiological parameters measured by the monitors 112 illustrated anddescribed in more detail with reference to FIG. 1 . The feature vector212 includes features 212 a, 212 b, . . . , 212 n. The featureextraction module 208 reduces the redundancy in the input data 204,e.g., repetitive data values, to transform the input data 204 into thereduced set of features 212, e.g., features 212 a, 212 b, . . . , 212 n.The feature vector 212 contains the relevant information from the inputdata 204, such that events or data value thresholds of interest can beidentified by the ML model 216 by using this reduced representation. Insome example embodiments, the following dimensionality reductiontechniques are used by the feature extraction module 208: independentcomponent analysis, Isomap, kernel principal component analysis (PCA),latent semantic analysis, partial least squares, PCA, multifactordimensionality reduction, nonlinear dimensionality reduction,multilinear PCA, multilinear subspace learning, semidefinite embedding,autoencoder, and deep feature synthesis.

In alternate embodiments, the ML model 216 performs deep learning (alsoknown as deep structured learning or hierarchical learning) directly onthe input data 204 to learn data representations, as opposed to usingtask-specific algorithms. In deep learning, no explicit featureextraction is performed; the features 212 are implicitly extracted bythe ML system 200. For example, the ML model 216 can use a cascade ofmultiple layers of nonlinear processing units for implicit featureextraction and transformation. Each successive layer uses the outputfrom the previous layer as input. The ML model 216 can thus learn insupervised (e.g., classification) and/or unsupervised (e.g., patternanalysis) modes. The ML model 216 can learn multiple levels ofrepresentations that correspond to different levels of abstraction,wherein the different levels form a hierarchy of concepts. In thismanner, the ML model 216 can be configured to differentiate features ofinterest from background features.

In alternative example embodiments, the ML model 216, e.g., in the formof a CNN generates the output 224, without the need for featureextraction, directly from the input data 204. The output 224 is providedto the computer device 228 or the console 108 illustrated and describedin more detail with reference to FIG. 1 . The computer device 228 is aserver, computer, tablet, smartphone, smart speaker, etc., implementedusing components of the example computer system 300 illustrated anddescribed in more detail with reference to FIG. 3 . In some embodiments,the steps performed by the ML system 200 are stored in memory on thecomputer device 228 for execution. In other embodiments, the output 224is displayed on the high-definition monitors 124 illustrated anddescribed in more detail with reference to FIG. 1 .

A CNN is a type of feed-forward artificial neural network in which theconnectivity pattern between its neurons is inspired by the organizationof a visual cortex. Individual cortical neurons respond to stimuli in arestricted region of space known as the receptive field. The receptivefields of different neurons partially overlap such that they tile thevisual field. The response of an individual neuron to stimuli within itsreceptive field can be approximated mathematically by a convolutionoperation. CNNs are based on biological processes and are variations ofmultilayer perceptrons designed to use minimal amounts of preprocessing.

The ML model 216 can be a CNN that includes both convolutional layersand max pooling layers. The architecture of the ML model 216 can be“fully convolutional,” which means that variable sized sensor datavectors can be fed into it. For all convolutional layers, the ML model216 can specify a kernel size, a stride of the convolution, and anamount of zero padding applied to the input of that layer. For thepooling layers, the model 216 can specify the kernel size and stride ofthe pooling.

In some embodiments, the ML system 200 trains the ML model 216, based onthe training data 220, to correlate the feature vector 212 to expectedoutputs in the training data 220. As part of the training of the MLmodel 216, the ML system 200 forms a training set of features andtraining labels by identifying a positive training set of features thathave been determined to have a desired property in question, and, insome embodiments, forms a negative training set of features that lackthe property in question.

The ML system 200 applies ML techniques to train the ML model 216, thatwhen applied to the feature vector 212, outputs indications of whetherthe feature vector 212 has an associated desired property or properties,such as a probability that the feature vector 212 has a particularBoolean property, or an estimated value of a scalar property. The MLsystem 200 can further apply dimensionality reduction (e.g., via lineardiscriminant analysis (LDA), PCA, or the like) to reduce the amount ofdata in the feature vector 212 to a smaller, more representative set ofdata.

The ML system 200 can use supervised ML to train the ML model 216, withfeature vectors of the positive training set and the negative trainingset serving as the inputs. In some embodiments, different ML techniques,such as linear support vector machine (linear SVM), boosting for otheralgorithms (e.g., AdaBoost), logistic regression, naïve Bayes,memory-based learning, random forests, bagged trees, decision trees,boosted trees, boosted stumps, neural networks, CNNs, etc., are used. Insome example embodiments, a validation set 232 is formed of additionalfeatures, other than those in the training data 220, which have alreadybeen determined to have or to lack the property in question. The MLsystem 200 applies the trained ML model 216 to the features of thevalidation set 232 to quantify the accuracy of the ML model 216. Commonmetrics applied in accuracy measurement include: Precision and Recall,where Precision refers to a number of results the ML model 216 correctlypredicted out of the total it predicted, and Recall is a number ofresults the ML model 216 correctly predicted out of the total number offeatures that had the desired property in question. In some embodiments,the ML system 200 iteratively re-trains the ML model 216 until theoccurrence of a stopping condition, such as the accuracy measurementindication that the ML model 216 is sufficiently accurate, or a numberof training rounds having taken place. The validation set 232 caninclude confirmed tissue states, tissue conditions, diagnoses, and/orcombinations thereof. This allows the detected values (e.g., valuesdiscussed in connection with FIG. 9 or other values disclosed herein) tobe validated using the validation sets 232. The validation sets 232 canbe generated based on analysis to be performed.

FIG. 3 is a block diagram illustrating an example computer system, inaccordance with one or more embodiments. Components of the examplecomputer system 300 can be used to implement the monitors 112, theconsole 108, or the EHR database 106 illustrated and described in moredetail with reference to FIG. 1 . In some embodiments, components of theexample computer system 300 are used to implement the ML system 200illustrated and described in more detail with reference to FIG. 2 . Atleast some operations described herein can be implemented on thecomputer system 300.

The computer system 300 can include one or more central processing units(“processors”) 302, main memory 306, non-volatile memory 310, networkadapters 312 (e.g., network interface), video displays 318, input/outputdevices 320, control devices 322 (e.g., keyboard and pointing devices),drive units 324 including a storage medium 326, and a signal generationdevice 320 that are communicatively connected to a bus 316. The bus 316is illustrated as an abstraction that represents one or more physicalbuses and/or point-to-point connections that are connected byappropriate bridges, adapters, or controllers. The bus 316, therefore,can include a system bus, a Peripheral Component Interconnect (PCI) busor PCI-Express bus, a HyperTransport or industry standard architecture(ISA) bus, a small computer system interface (SCSI) bus, a universalserial bus (USB), IIC (I2C) bus, or an Institute of Electrical andElectronics Engineers (IEEE) standard 1394 bus (also referred to as“Firewire”).

The computer system 300 can share a similar computer processorarchitecture as that of a desktop computer, tablet computer, personaldigital assistant (PDA), mobile phone, game console, music player,wearable electronic device (e.g., a watch or fitness tracker),network-connected (“smart”) device (e.g., a television or home assistantdevice), virtual/augmented reality systems (e.g., a head-mounteddisplay), or another electronic device capable of executing a set ofinstructions (sequential or otherwise) that specify action(s) to betaken by the computer system 300.

While the main memory 306, non-volatile memory 310, and storage medium326 (also called a “machine-readable medium”) are shown to be a singlemedium, the term “machine-readable medium” and “storage medium” shouldbe taken to include a single medium or multiple media (e.g., acentralized/distributed database and/or associated caches and servers)that store one or more sets of instructions 328. The term“machine-readable medium” and “storage medium” shall also be taken toinclude any medium that is capable of storing, encoding, or carrying aset of instructions for execution by the computer system 300.

In general, the routines executed to implement the embodiments of thedisclosure can be implemented as part of an operating system or aspecific application, component, program, object, module, or sequence ofinstructions (collectively referred to as “computer programs”). Thecomputer programs typically include one or more instructions (e.g.,instructions 304, 308, 328) set at various times in various memory andstorage devices in a computer device. When read and executed by the oneor more processors 302, the instruction(s) cause the computer system 300to perform operations to execute elements involving the various aspectsof the disclosure.

Moreover, while embodiments have been described in the context of fullyfunctioning computer devices, those skilled in the art will appreciatethat the various embodiments are capable of being distributed as aprogram product in a variety of forms. The disclosure applies regardlessof the particular type of machine or computer-readable media used toactually effect the distribution.

Further examples of machine-readable storage media, machine-readablemedia, or computer-readable media include recordable-type media such asvolatile and non-volatile memory devices 310, floppy and other removabledisks, hard disk drives, optical discs (e.g., Compact Disc Read-OnlyMemory (CD-ROMS), Digital Versatile Discs (DVDs)), and transmission-typemedia such as digital and analog communication links.

The network adapter 312 enables the computer system 300 to mediate datain a network 314 with an entity that is external to the computer system300 through any communication protocol supported by the computer system300 and the external entity. The network adapter 312 can include anetwork adapter card, a wireless network interface card, a router, anaccess point, a wireless router, a switch, a multilayer switch, aprotocol converter, a gateway, a bridge, a bridge router, a hub, adigital media receiver, and/or a repeater.

The network adapter 312 can include a firewall that governs and/ormanages permission to access proxy data in a computer network and tracksvarying levels of trust between different machines and/or applications.The firewall can be any number of modules having any combination ofhardware and/or software components able to enforce a predetermined setof access rights between a particular set of machines and applications,machines and machines, and/or applications and applications (e.g., toregulate the flow of traffic and resource sharing between theseentities). The firewall can additionally manage and/or have access to anaccess control list that details permissions including the access andoperation rights of an object by an individual, a machine, and/or anapplication, and the circumstances under which the permission rightsstand.

FIG. 4A is a block diagram illustrating an example robotic surgicalsystem 400, in accordance with one or more embodiments. The roboticsurgical system 400 is the same as or similar to the robotic surgicalsystem 160 illustrated and described in more detail with reference toFIG. 1 . The robotic surgical system 400 can include components andfeatures discussed in connection with FIGS. 1-3 and 4B-5 . For example,the robotic surgical system 400 can include a console 420 with featuresof the console 108 of FIG. 1 . Likewise, the components and features ofFIG. 4A can be included or used with other embodiments disclosed herein.For example, the description of the input devices of FIG. 4A appliesequally to other input devices (e.g., input devices 166 of FIG. 1 ).

The robotic surgical system 400 includes a user device or console 420(“console 420”), a surgical robot 440, and a computer or data system450. The console 420 can be operated by a surgeon and can communicatewith components in an operating room 402, remote devices/servers, anetwork 404, or databases (e.g., database 106 of FIG. 1 ) via thenetwork 404. The robotic surgical system 400 can include surgicalcontrol software and can include a guidance system (e.g., ML guidancesystem, AI guidance system, etc.), surgical planning software, eventdetection software, surgical tool software, etc. or other featuresdisclosed herein to perform surgical step(s) or procedures or implementsteps of processes discussed herein.

The user 421 can use the console 420 to view and control the surgicalrobot 440. The console 420 can be communicatively coupled to one or morecomponents disclosed herein and can include input devices operated byone, two, or more users. The input devices can be hand-operatedcontrols, but can alternatively, or in addition, include controls thatcan be operated by other parts of the user's body, such as, but notlimited to, foot pedals. The console 420 can include a clutch pedal toallow the user 421 to disengage one or more sensor-actuator componentsfrom control by the surgical robot 440. The console 420 can also includedisplay or output so that the one of more users can observe the patientbeing operated on, or the product being assembled, for example. In someembodiments, the display can show images, such as, but not limited tomedical images, video, etc. For surgical applications, the images couldinclude, but are not limited to, real-time optical images, real-timeultrasound, real-time OCT images and/or other modalities, or couldinclude pre-operative images, such as MM, CT, PET, etc. The variousimaging modalities can be selectable, programmed, superimposed, and/orcan include other information superimposed in graphical and/or numericalor symbolic form.

The robotic surgical system 400 can include multiple consoles 420 toallow multiple users to simultaneously or sequentially perform portionsof a surgical procedure. The term “simultaneous” herein refers toactions performed at the same time or in the same surgical step. Thenumber and configuration of consoles 420 can be selected based on thesurgical procedure to be performed, number and configurations ofsurgical robots, surgical team capabilities, or the like.

FIG. 4B illustrates an example console 420 of the robotic surgicalsystem 400 of FIG. 4A, in accordance with one or more embodiments. Theconsole 420 includes hand-operated input devices 424, 426, illustratedheld by the user's left and right hands 427, 428, respectively. A viewer430 includes left and right eye displays 434, 436. The user can view,for example, the surgical site, instruments 437, 438, or the like. Theuser's movements of the input devices 424, 426 can be translated inreal-time to, for example, mimic the movement of the user on the viewer430 and display (e.g., the high definition monitors 124 of FIG. 1 ) andwithin the patient's body while the user can be provided with output,such as alerts, notifications, and information. The information caninclude, without limitation, surgical or implantation plans, patientvitals, modification to surgical plans, values, scores, predictions,simulations, and other output, data, and information disclosed herein.The console 420 can be located at the surgical room or at a remotelocation.

The viewer 430 can display at least a portion of a surgical plan,including maps (e.g., tissue maps, bone tissue maps, tissue densitymaps, diseased tissue maps, tissue condition maps, etc.), past andfuture surgical steps, patient monitor readings (e.g., vitals), surgicalroom information (e.g., available team members, available surgicalequipment, surgical robot status, or the like), images (e.g.,pre-operative images, images from simulations, real-time images,instructional images, etc.), and other surgical assist information. Insome embodiments, the viewer 430 can be a VR/AR headset, display, or thelike. The robotic surgical system 400, illustrated and described in moredetail with reference to FIG. 4A, can further include multiple viewers430 so that multiple members of a surgical team can view the surgicalprocedure. The number and configuration of the viewers 430 can beselected based on the configuration and number of surgical robots.

Referring again to FIG. 4A, the surgical robot 440 can include one ormore controllers, computers, sensors, arms, articulators, joints, links,grippers, motors, actuators, imaging systems, effector interfaces, endeffectors, or the like. For example, a surgical robot with a high numberof degrees of freedom can be used to perform complicated procedureswhereas a surgical robot with a low number of degrees of freedom can beused to perform simple procedures. The configuration (e.g., number ofarms, articulators, degrees of freedom, etc.) and functionality of thesurgical robot 440 can be selected based on the procedures to beperformed.

The surgical robot 440 can operate in different modes selected by auser, set by the surgical plan, and/or selected by the robotic surgicalsystem 400. In some procedures, the surgical robot 440 can remain in thesame mode throughout a surgical procedure. In other procedures, thesurgical robot 440 can be switched between modes any number of times.The configuration, functionality, number of modes, and type of modes canbe selected based on the desired functionality and user control of therobotic surgical system 400. The robotic surgical system 400 can switchbetween modes based on one or more features, such as triggers,notifications, warnings, events, etc. Different example modes arediscussed below. A trigger can be implemented in software to execute ajump to a particular instruction or step of a program. A trigger can beimplemented in hardware, e.g., by applying a pulse to a trigger circuit.

In a user control mode, a user 421 controls, via the console 420,movement of the surgical robot 440. The user's movements of the inputdevices can be translated in real-time into movement of end effectors452 (one identified).

In a semi-autonomous mode, the user 421 controls selected steps and thesurgical robot 440 autonomously performs other steps. For example, theuser 421 can control one robotic arm to perform one surgical step whilethe surgical robot 440 autonomously controls one or more of the otherarms to concurrently perform another surgical step. In another example,the user 421 can perform steps suitable for physician control. Aftercompletion, the surgical robot 440 can perform steps involvingcoordination between three or more robotic arms, thereby enablingcomplicated procedures. For example, the surgical robot 440 can performsteps involving four or five surgical arms, each with one or more endeffectors 452.

In an autonomous mode, the surgical robot 440 can autonomously performsteps under the control of the data system 450. The robotic surgicalsystem 400 can be pre-programmed with instructions for performing thesteps autonomously. For example, command instructions can be generatedbased on a surgical plan. The surgical robot 440 autonomously performssteps or the entire procedure. The user 421 and surgical team canobserve the surgical procedure to modify or stop the procedure.Advantageously, complicated procedures can be autonomously performedwithout user intervention to enable the surgical team to focus andattend to other tasks. Although the robotic surgical system 400 canautonomously perform steps, the surgical team can provide information inreal-time that is used to continue the surgical procedure. Theinformation can include physician input, surgical team observations, andother data input.

The robotic surgical system 400 can also adapt to the user control tofacilitate completion of the surgical procedure. In some embodiments,the robotic surgical system 400 can monitor, via one or more sensors, atleast a portion of the surgical procedure performed by the surgicalrobot 440. The robotic surgical system 400 can identify an event, suchas a potential adverse surgical event, associated with a roboticallyperformed surgical task. For example, a potential adverse surgical eventcan be determined based on acquired monitoring data and information forthe end effector, such as surgical tool data from a medical devicereport, database, manufacturer, etc. The robotic surgical system 400 canperform one or more actions based on the identified event. The actionscan include, without limitation, modification of the surgical plan toaddress the potential adverse surgical event, thereby reducing the riskof the event occurring. The adverse surgical event can include one ormore operating parameters approaching respective critical thresholds, asdiscussed in connection with FIG. 12 . The adverse surgical events canbe identified using a machine learning model trained using, for example,prior patient data, training sets (e.g., tool data), etc.

In some embodiments, the robotic surgical system 400 determines whethera detected event (e.g., operational parameters outside a target range orexceeding a threshold, etc.) is potentially an adverse surgical eventbased on one or more criteria set by the robotic surgical system 400,user, or both. The adverse surgical event can be an adversephysiological event of the patient, surgical robotic malfunction,surgical errors, or other event that can adversely affect the patient orthe outcome of the surgery. Surgical events can be defined and inputtedby the user, surgical team, healthcare provider, manufacturer of therobotic surgery system, or the like.

The robotic surgical system 400 can take other actions in response toidentification of an event. If the robotic surgical system 400identifies an end effector malfunction or error, the robotic surgicalsystem 400 can stop usage of the end effector and replace themalfunctioning component (e.g., surgical tool or equipment) to completethe procedure. The robotic surgical system 400 can monitor hospitalinventory, available resources in the surgical room 402, time to acquireequipment (e.g., time to acquire replacement end effectors, surgicaltools, or other equipment), and other information to determine how toproceed with surgery. The robotic surgical system 400 can generatemultiple proposed surgical plans for continuing with the surgicalprocedure. The user and surgical team can review the proposed surgicalplans to select an appropriate surgical plan. The robotic surgicalsystem 400 can modify a surgical plan with one or more correctivesurgical steps based on identified surgical complications, sensorreadings, or the like.

The robotic surgical system 400 can retrieve surgical system informationfrom a database to identify events. The database can describe, forexample, maintenance of the robotic surgery system, specifications ofthe robotic surgery system, specifications of end effectors, surgicalprocedure information for surgical tools, consumable informationassociated with surgical tools, operational programs and parameters forsurgical tools, monitoring protocols for surgical tools, or the like.The robotic surgical system 400 can use other information in databasesdisclosed herein to generate rules for triggering actions, identifyingwarnings, defining events, or the like. Databases can be updated withdata (e.g., intraoperative data collected during the surgical procedure,simulation data, etc.) to intraoperatively adjust surgical plans,collect data for ML/AI training sets, or the like. Data from on-site andoff-site simulations (e.g., pre-, or post-operative virtual simulations,simulations using models, etc.) can be generated and collected.

The surgical robot 440 can include robotic arms 451 (one identified)with integrated or removable end effectors 452 (one identified). The endeffectors 452 can include, without limitation, bone mapper devices,imagers (e.g., cameras, optical guides, etc.), robotic grippers,instrument holders, cutting instruments (e.g., cutters, scalpels, or thelike), drills, cannulas, reamers, rongeurs, scissors, clamps, or otherequipment or surgical tools disclosed herein. In some embodiments, theend effectors can be reusable or disposable surgical tools. The numberand configuration of end effectors can be selected based on theconfiguration of the robotic system, procedure to be performed, surgicalplan, etc. Imaging and viewing technologies can integrate with thesurgical robot 440 to provide more intelligent and intuitive results.

The data system 450 can improve surgical planning, monitoring (e.g., viathe display 422), data collection, surgical robotics/navigation systems,intelligence for selecting instruments, implants, etc. The data system450 can execute, for example, surgical control instructions or programsfor a guidance system (e.g., ML guidance system, AI guidance system,etc.), surgical planning programs, event detection programs, surgicaltool programs, etc. For example, the data system 450 can increaseprocedure efficiency and reduce surgery duration by providinginformation insertion paths, surgical steps, or the like. The datasystem 450 can be incorporated into or include other components andsystems disclosed herein. The display 422 can display, for example, adiagnosis of tissue, maps, surgical plans, etc. For example, the display422 can display a diagnostic map showing, for example, a bone 423(discussed in more detail below with respect to FIG. 10 ), regions ofinterest (e.g., zones of diseased tissue, regions of tissue withspecific characteristic(s), margins, etc.), features of interest,anatomical elements (e.g., cartilage, soft tissue, etc.), or the like.In some embodiments, the diagnostic map can include tissue density,tissue state, identified disease tissue, or the like. The system 402 canuse the displayed data to perform one or more surgical steps. A user canview the display 422 to confirm the position of the tissue during theprocedure.

The robotic surgical system 400 can be used to perform open procedures,minimally invasive procedures, such as laparoscopic surgeries,non-robotic laparoscopic/abdominal surgery, retroperitoneoscopy,arthroscopy, pelviscopy, nephroscopy, cystoscopy, cisternoscopy,sinoscopy, hysteroscopy, urethroscopy, and the like. The methods,components, apparatuses, and systems can be used with many differentsystems for conducting robotic or minimally invasive surgery. Oneexample of a surgical system and surgical robots which can incorporatemethods and technology is the DAVINCI™ system available from IntuitiveSurgical, Inc.™ of Mountain View, California. However, other surgicalsystems, robots, and apparatuses can be used.

The robotic surgical system 400 can perform one or more simulationsusing selected entry port placements and/or robot positions, to allow asurgeon or other user to practice procedures. The practice session canbe used to generate, modified, or select a surgical plan. In someembodiments, the system can generate a set of surgical plans forphysician consideration. The physician can perform practice sessions foreach surgical plan to determine and select a surgical plan to beimplemented. In some embodiments, the systems disclosed herein canperform virtual surgeries to recommend a surgical plan. The physiciancan review the virtual simulations to accept or reject the recommendedsurgical plan. The physician can modify surgical plans pre-operative orintraoperatively.

Embodiments can provide a means for mapping the surgical path forneurosurgery procedures that minimize damage through artificialintelligence mapping. The software for artificial intelligence istrained to track the least destructive pathway. The physician can makean initial incision based on a laser marking on the skin thatilluminates the optimal site. Next, a robot can make a small hole andinsert surgical equipment (e.g., guide wires, cannulas, etc.) thathighlights the best pathway. This pathway minimizes the amount of tissuedamage that occurs during surgery. Mapping can also be used to identifyone or more insertion points associated with a surgical path. Mappingcan be performed before treatment, during treatment, and/or aftertreatment. For example, pretreatment and posttreatment mapping can becompared by the surgeon and/or ML/AI system. The comparison can be usedto determine next steps in a procedure and/or further train the ML/AIsystem.

FIG. 5 is a schematic block diagram illustrating subcomponents of therobotic surgical system 400 of FIG. 4A in accordance with embodiment ofthe present technology. The data system 450 has one or more processors504, a memory 506, input/output devices 508, and/or subsystems and othercomponents 510. The processor 504 can perform any of a wide variety ofcomputing processing, image processing, robotic system control, plangeneration or modification, and/or other functions. Components of thedata system 450 can be housed in a single unit (e.g., within a hospitalor surgical room) or distributed over multiple, interconnected units(e.g., though a communications network). The components of the datasystem 450 can accordingly include local and/or devices.

As illustrated in FIG. 5 , the processor 504 can include a plurality offunctional modules 512, such as software modules, for execution by theprocessor 504. The various implementations of source code (i.e., in aconventional programming language) can be stored on a computer-readablestorage medium or can be embodied on a transmission medium in a carrierwave. The modules 512 of the processor 504 can include an input module514, a database module 516, a process module 518, an output module 520,and, optionally, a display module 524 for controlling the display.

In operation, the input module 514 accepts an operator input 524 via theone or more input devices, and communicates the accepted information orselections to other components for further processing. The databasemodule 516 organizes plans (e.g., robotic control plans, surgical plans,etc.), records (e.g., maintenance records, patient records, historicaltreatment data, etc.), surgical equipment data (e.g., instrumentspecifications), control programs, and operating records and otheroperator activities, and facilitates storing and retrieving of theserecords to and from a data storage device (e.g., internal memory 506,external databases, etc.). Any type of database organization can beutilized, including a flat file system, hierarchical database,relational database, distributed database, etc.

In the illustrated example, the process module 518 can generate controlvariables based on sensor readings 526 from sensors (e.g., end effectorsensors of the surgical robot 440, bone mapper devices, patientmonitoring equipment, etc.), operator input 524 (e.g., input from thesurgeon console 420 and/or other data sources), and the output module520 can communicate operator input to external computing devices andcontrol variables to controllers. The display module 522 can beconfigured to convert and transmit processing parameters, sensorreadings 526, output signals 528, input data, treatment profiles andprescribed operational parameters through one or more connected displaydevices, such as a display screen, touchscreen, printer, speaker system,etc.

In various embodiments, the processor 504 can be a standard centralprocessing unit or a secure processor. Secure processors can bespecial-purpose processors (e.g., reduced instruction set processor)that can withstand sophisticated attacks that attempt to extract data orprogramming logic. The secure processors cannot have debugging pins thatenable an external debugger to monitor the secure processor's executionor registers. In other embodiments, the system can employ a securefield-programmable gate array, a smartcard, or other secure devices.

The memory 506 can be standard memory, secure memory, or a combinationof both memory types. By employing a secure processor and/or securememory, the system can ensure that data and instructions are both highlysecure and sensitive operations such as decryption are shielded fromobservation. In various embodiments, the memory 506 can be flash memory,secure serial EEPROM, secure field-programmable gate array, or secureapplication-specific integrated circuit. The memory 506 can storeinstructions for causing the surgical robot 440 to perform actsdisclosed herein.

The input/output device 508 can include, without limitation, atouchscreen, a keyboard, a mouse, a stylus, a push button, a switch, apotentiometer, a scanner, an audio component such as a microphone, orany other device suitable for accepting user input and can also includeone or more video monitors, a medium reader, an audio device such as aspeaker, any combination thereof, and any other device or devicessuitable for providing user feedback. For example, if an applicatormoves an undesirable amount during a treatment session, the input/outputdevice 508 can alert the subject and/or operator via an audible alarm.The input/output device 508 can be a touch screen that functions as bothan input device and an output device.

The data system 450 can output instructions to command the surgicalrobot 440 and communicate with one or more databases 2600. The surgicalrobot 440 or other components disclosed herein can communicate to sendcollected data (e.g., sensor readings, instrument data, surgical robotdata, etc.) to the database 500. This information can be used to, forexample, create new training data sets, generate plans, perform futuresimulations, post-operatively analyze surgical procedures, or the like.The data system 450 can be incorporated, used with, or otherwiseinteract with other databases, systems, and components disclosed herein.In some embodiments, the data system 450 can be incorporated into thesurgical robot 440 or other systems disclosed herein. In someembodiments, the data system 450 can be located at a remote location andcan communicate with a surgical robot via one or more networks. Forexample, the data system 450 can communicate with a hospital via anetwork, such as a wide area network, a cellular network, etc. One ormore local networks at the hospital can establish communication channelsbetween surgical equipment within the surgical room.

A surgical program or plan (“surgical plan”) can include, withoutlimitation, patient data (e.g., pre-operative images, medical history,physician notes, etc.), imaging programs, surgical steps, mode switchingprograms, criteria, goals, or the like. The imaging programs caninclude, without limitation, AR/VR programs, identification programs(e.g., fiducial identification programs, tissue identification programs,target tissue identification programs, etc.), image analysis programs,or the like. Surgical programs can define surgical procedures or aportion thereof. For example, surgical programs can include end effectorinformation, positional information, surgical procedure protocols,safety settings, surgical robot information (e.g., specifications, usagehistory, maintenance records, performance ratings, etc.), order ofsurgical steps, acts for a surgical step, feedback (e.g., hapticfeedback, audible feedback, etc.), or the like. The mode switchingprograms can be used to determine when to switch the mode of operationof the surgical robot 440. For example, mode switching programs caninclude threshold or configuration settings for determining when toswitch the mode of operation of the surgical robot 440. Example criteriacan include, without limitation, thresholds for identifying events, datafor evaluating surgical steps, monitoring criteria, patient healthcriteria, physician preference, or the like. The goals can includeintraoperative goals, post-operative goals (e.g., target outcomes,metrics, etc.), goal rankings, etc. Monitoring equipment or the surgicalteam can determine goal progress, whether a goal has been achieved, etc.If an intraoperative goal is not met, the surgical plan can be modifiedin real-time so that, for example, the post-operative goal is achieved.The post-operative goal can be redefined intraoperatively in response toevents, such as surgical complications, unplanned changes to patient'svitals, etc.

The surgical plan can also include healthcare information, surgical teaminformation, assignments for surgical team members, or the like. Thehealthcare information can include surgical room resources, hospitalresources (e.g., blood banks, standby services, available specialists,etc.), local or remote consultant availability, insurance information,cost information (e.g., surgical room costs, surgical team costs, etc.).

The systems disclosed herein can generate pre-operative plans andsimulation plans. Pre-operative plans can include scheduling ofequipment, surgical room, staff, surgical teams, and resources forsurgery. The systems can retrieve information from one or more databasesto generate the pre-operative plan based on physician input, insuranceinformation, regulatory information, reimbursements, patient medicalhistory, patient data, or the like. Pre-operative plans can be used togenerate surgical plans, cost estimates, scheduling of consultants andremote resources, or the like. For example, a surgical plan can begenerated based on available resources scheduled by the pre-operativeplans. If a resource becomes unavailable, the surgical plan can beadjusted for the change in resources. The healthcare provider can bealerted if additional resources are recommended. The systems disclosedherein can generate simulation plans for practicing surgical procedures.On approval, a surgeon can virtually simulate a procedure using aconsole or another simulation device. Plans (e.g., surgical plans,implantation plans, etc.) can be generated and modified based on thesurgeon's performance and simulated outcome.

The systems disclosed herein can generate post-operative plans forevaluating surgical outcomes, developing physical therapy and/or rehabprograms and plans, etc. The post-operative plans can be modified by thesurgical team, primary care provider, and others based on the recoveryof the patient. In some embodiments, systems generate pre-operativeplans, surgical plans, and post-operative plans prior to beginning asurgical procedure. The system then modifies one or more or the plans asadditional information is provided. For example, one or more steps ofthe methods discussed herein can generate data that is incorporated intothe plan. ML data sets to be incorporated into the plan generate a widerange of variables to be considered when generating plans. Plans can begenerated to optimize patient outcome, reduce or limit the risk ofsurgical complications, mitigate adverse events, manage costs forsurgical procedures, reduce recovery time, or the like. The healthcareprovider can modify how plans are generated over time to furtheroptimize based on one or more criteria.

FIG. 6 is a block diagram of a system 600 for performing real-timeanalysis of bone tissue in accordance with some embodiments of thepresent technology. The system 600 can be incorporated into or used withtechnology discussed in connection with FIGS. 1-5 . For example, one ormore components of the analysis equipment 602 can be incorporated intothe operating room 102 discussed in FIG. 1 . By way of another example,the user interface 606 of the system 600 can be part of the interface420 discussed in connection with FIG. 4B. Output from the system 600 canbe transmitted to the controller 450 at FIG. 5 and/or various othercomponents disclosed herein. Accordingly, the system 600 can beincorporated into robotic surgery systems, or utilized to perform manualprocedures or to perform other procedures disclosed herein.

In some embodiments, the system 600 is configured to perform in-situtesting and analysis of samples of a target tissue (e.g., bone tissue,scar tissue, pathogenic tissue, and/or other suitable bodily tissue).Additionally, or alternatively, the system 600 can perform in-situtesting and analysis of surrounding tissues without requiring a biopsyor bone sample to be taken or removed from a patient. To do so, forexample, the system 600 can include a sampling unit capable of samplingbone tissue in real-time by subjecting the target tissue and/orsurrounding tissue (also referred to collectively herein as the “tissuesample”) to at least one test. The test(s) can include multi-wavelengthphotoacoustic measurements (MWPM), ultrasound measurements, one or morex-ray measurements, other optical measurements, one or more targetedultrasound measurements, and/or various other suitable measurements. Insome embodiments, the system 600 performs one or more multimodalityanalyses in which one or more multi-sensing devices perform(sequentially or concurrently) a plurality of tests, such as opticaltests, acoustic tests, photoacoustic tests, combinations thereof, or thelike. The tests can be performed during one or more scans of the tissuesample. In a single scan test, the system 600 can concurrently performmultiple tests while moving along the tissue sample. In multiple scantest, the system 600 can sequentially perform tests during correspondingscans and/or can concurrently perform multiple tests during each scan.The system 600 can perform different imaging or scanning protocols basedon the analysis to be performed.

The system 600 can then facilitate communication with a robotic surgicalsystem, doctor, surgeon, or other medical professional by providingresults (e.g., multiplexed data, raw data, visualizations of the data,and the like) from the test(s) in real-time. Further, the system 600 cancombine the results from the test(s) (sometimes referred to herein asthe “scan”) to provide a diagnosis of the tissue sample. In surgicalprocedures, the results can be automatically transmitted to a roboticsurgical system that analyses the results to perform one or moresurgical steps. The robotic surgical system can request additionalinformation from the system 600 to, for example, complete a surgicalstep, confirm completion of a surgical step, plan a surgical step, plana series of surgical steps, or the like. For example, the surgicalsystem 402 at FIG. 4A can receive results from the system 600 to performa MWPM-guided robotic surgical step. In some embodiments, the resultscan be displayed via display 422 for viewing by the surgical team.Additionally, or alternatively, the results can be viewable via console420 by a user 421 while, for example, monitoring or performing one ormore surgical steps.

As discussed in more detail below, the system 600 can perform variousmeasurements to obtain a tissue density value (e.g., a bone mineraldensity (BMD) value), a MWMP value, a μ3 value, one or more x-rayimages, CT scan values, and/or any other suitable values during the scan(sometimes referred to collectively as “values from the scan”), thenidentify relevant reference cases in a database (third party or local).The system 600 can then generate the diagnosis based on the values fromthe scan, predetermined threshold values for each of the measuredvalues, and/or the diagnoses in the identified reference cases.Additionally, or alternatively, the system 600 can generate a 3Danalysis of the tissue sample using the values from the scan and createa map (e.g., a visualization similar to a detailed 3D x-ray from medicalimagery) of the tissue sample. The 3D analysis can include a variety oftissue analytics generated using a tissue mapper device, automaticallygenerated diagnostic information and/or suggested diagnoses, and thelike. The system 600 can then provide the doctor (or other medicalprofessional) with access to the map to provide a diagnosis and/ormedical recommendations based on the map.

As illustrated in FIG. 6 , the system 600 can include operating roomreal-time analysis equipment component 602 (sometimes referred to hereinas the “operating room 602”) that is configured to perform real-timetesting and analysis of the tissue sample of the patient. In variousembodiments, the target tissue can include, but not be limited to, bonetissue for detection of osteoporosis, osteopenia, bone cancer, and thelike; and/or pathogenic tissue for detection of osteomyelitis, othercancerous cells, inflammation, and the like. In the illustratedembodiment, the operating room 602 includes a processor 604 configuredto perform a plurality of functions related to the operating room 602,as well as a user interface 606, a power supply 608, a tissue mapperdevice such as bone mapper device 609, a pathology component 612, amemory unit 614, and a communication interface 620 all operativelycoupled to the processor 604. The bone mapper device 609 can include oneor more hybrid imaging components, here a hybrid MWPM component 610, andimaging devices 611. In some embodiments, the operating room 602includes the features discussed in connection with FIGS. 1-5 .

The processor 604 can be configured to perform a plurality of functionsrelated to the operating room 602, for example, based on a set ofinstructions and/or received inputs. In various embodiments, thefunctions can include commanding the hybrid MWMP component 610 and/orthe pathology component 612 to perform measurements to generate sampledata, receiving the sample data, retrieving previous sample datarelevant to an individual patient, retrieving previous sample datarelated to one or more reference cases, evaluating the sample data,generating the 3D map based one the sample data, and the like. In someembodiments, the set of instructions is stored in the memory unit 614.In some embodiments, the set of instructions causes the processor 604 toautomatically perform one or more scans over and/or an analysis of thetissue sample using any of the components of the operating room 602.

The user interface 606 can display the results of the scan(s) (e.g.,measurements taken using the hybrid MWMP component 610, the diagnosis ofthe tissue sample, the 3D map, and the like) for the doctor, surgeon,and/or other medical professional. In some embodiments, the operatingroom 602 includes a second user interface (not shown, sometimes alsoreferred to herein as an “auxiliary interface”) coupled to an imagingdevice (not shown) that is directed at the tissue sample. The seconduser interface can continuously display actions performed during testingand analysis of the tissue sample. In various such embodiments, theimaging device can include a camera, a video recorder, an optical imagesensor, and the like. In some embodiments, the imaging device isdirected toward the tissue sample to capture a real-time image which canbe fed to the auxiliary user interface for the surgeon to analyze thetarget tissue and surrounding tissues while the hybrid MWMP component610 scans the tissue sample (e.g., takes measurements of the tissuesample). In some embodiments, the auxiliary user interface allows thedoctor to provide instructions to the processor 604 during the scan tocontrol one or more components of the operating room 602.

The power supply 608 can be operatively coupled to the processor 604and/or any other component of the operating room 602 to drive each ofthe components. In some embodiments, the power supply 608 is an internalpower source for the operating room 602. Purely by way of example, thepower supply 608 can include a battery. In a specific, non-limitingexample, the power supply 608 can include a Lithium polymer battery(Li-Po), due to its lightweight, high discharge rate, and relativelyhigh capacity. In another example, the power supply 608 can include awall outlet (or other suitable component) coupled to an external powersource.

As illustrated in FIG. 6 , the processor 604 can be communicativelycoupled to the hybrid MWPM component 610 (e.g., via the communicationinterface 620 and/or any other suitable component) to direct the hybridMWPM component 610 to perform actions related to the analysis of thetissue sample. For example, the processor 604 can direct the hybrid MWPMcomponent 610 to perform a scan of the tissue sample. During the scan,the hybrid MWPM component 610, discussed in more detail below withrespect to FIG. 7 , can impart ultrasonic and/or photoacoustic wavesthrough the tissue sample and measure a response. As a result, thehybrid MWPM component 610 can determine a variety of values, such as amaterial density value (e.g., a value for bone material density (BMD)),an absorption spectrum (e.g., μ_(a)) value, and/or a MWPM value for oneor more portions of the tissue sample (sometimes referred tocollectively herein). The pathology component 612, in conjunction withthe processor 604, can then provide a pathology analysis of the datacollected by the hybrid MWPM component 610. Purely by way of example,the pathology component 612 and the processor 604 can identify a bonecondition in the tissue sample, such as osteoporosis, clinicalosteopenia, cancerous bone, normal bone, and/or any other condition.Because the system 600 diagnoses the tissue sample based on a variety ofvalues from the scan, the system is able to generate an accurate,real-time diagnosis of the tissue sample without requiring exposure topotentially harmful levels of radiation.

Additionally, or alternatively, the hybrid MWPM component 610 and/or theprocessor 604 can create the 3D map (or other visualization) of thetissue sample using images (e.g., still images, video, topology mapping,etc.) from the imaging device(s) 611, the values from the scan, and/orany other associated imagery (e.g., images from the imaging componentexternal to the bone mapper device 609). As discussed above, theprocessor 604 can then display the 3D map (e.g., on the user interface606) for review by a doctor or other medical professional. In someembodiments, image data captured by the imaging device 611 are used togenerate a 3D map of the sample tissue. Output from the hybrid MWPMcomponent 610 can be overlaid onto the image data. For example, thecaptured image can be a color still image and the output of the hybridMWPM component 610 can be colored to provide a false clear image. TheMWPM output can be readily identifiable by a user. This allows a user tosee overlaid output or analytics generated from the hybrid MWPMcomponent 610. In some procedures, the imaging device 611 can scan thetissue sample to generate a 3D image of the tissue sample. The outputfrom the hybrid MWPM component 610 can be overlaid onto the scan basedon the known positional information between the component 610 and theimaging device 611. This allows the acoustic-generated image data to bekeyed to the image data from the imaging device 611. Additionally, oralternatively, the processor 604 can display the diagnosis generated bythe system 600 and/or the values from the scan. The doctor can thendiagnose the tissue sample based on the displayed information. Becausethe system 600 is able to generate a 3D map and/or a variety of valuesfrom the scan, the system 600 can help improve the accuracy of thedoctors' real-time diagnosis without requiring exposure to potentiallyharmful levels of radiation. Purely by way of example, by includingvalues from both ultrasound and photoacoustic measurements, the system600 can create an accurate 3D map of an imaged bone that allows thedoctor to assess both density of the bone, amount of bone material, andmatrix structure of the bone. In a specific, non-limiting example, ifone of the values from the scan is above (or below) a predeterminedthreshold, the system 600 can set an initial diagnosis for review by thedoctor. In various embodiments, the predetermined threshold can be setby the system 600 based on a plurality of reference cases (e.g., when aconsistent threshold emerges), input by the doctor based on a practicepreference, and/or retrieved from a third party (e.g., another medicaldatabase, a trusted medical publication, and the like).

The memory unit 614 can store real-time data acquired from the patient(e.g., values from the scan, related medical data, other surgical data,patient history, and the like). For example, in the illustratedembodiment, the memory unit 614 includes an operating room real-timeequipment database 616 (“database 616”) that can store real-time dataacquired from the patient undergoing testing (e.g., bone testing) and/orany related analyses. In various embodiments, the real-time data caninclude the date of operation, surgery, and/or tests (sometimes referredto collectively herein as a “medical procedure” and/or a “surgicaloperation”); doctor identification and specialty; patient identificationand background medical information; tissue density values (e.g., BMDvalues), MWPM values, μ_(a) values, and/or any other suitable values fordiagnosis of tissue; data associated with a diagnosis and/or analysis;and/or metadata associated with the diagnosis and/or analysis.Additionally, or alternatively, the database 616 can store informationrelated to the physical location and/or orientation of the components ofthe system 600; a mapping analysis of the measurements to the tissuesample (e.g., information related to a picture location on the tissuesample); and the like. In a specific, non-limiting example, the database616 can store information showing that a patient named John Doe ishaving bone surgery on 21 Jun. 2020; the surgery is to be performed byDr. Smith at a surgical facility in Paris; that a sample bone (e.g., theulna) is selected to be monitored for diagnosis of osteoporosis usingthe hybrid MWPM component 610; that scans by the hybrid MWPM component610 measured a BMD value of −1.8, an MWPM value of 780 nm, and a μ_(a)value of 2; that the predetermined threshold MWPM value for John Doe wasset at 750 nm; and that Dr. Smith provided a diagnosis that John Doe issuffering from osteoporosis based on the values from the scan. Inanother specific example, the database 616 can store information thatthe imaging device, when directed towards the ulna bone (e.g., thetarget tissue) of John Doe, indicated a porous structure all over theulna. Osteoporosis may be indicated by a μ_(a) between 1 and 2 with adeviation of up to 0.5 at an MWPM between 700 nm and 950 nm whichcorrelates to a BMD value of less than −2.5, also known as a T-value. ABMD below −1 may indicate a low bone density. This would correlate to aCT scan of a bone with a bone density represented by less than 135Hounsfield Units (HUs). In some embodiments, the presence of the porousbone structure can help confirm the BMD value of −1.8 from the hybridMWPM component 610.

In some embodiments, the memory unit 614 is operably coupled to theprocessor 604 to store and retrieve historical data and/or referencedata (e.g., medical data related to one or more reference cases) inreal-time. Purely by way of example, as real-time scan values aregathered by the hybrid MWPM component 610, the memory unit 614 canretrieve reference cases from a third-party database with similar scanvalues (e.g., comparable BMD, MWPM, and/or μ_(a) values).

In the illustrated embodiment, memory unit 614 also includes a basemodule 618 that, stores, receives, and/or retrieves information relatedto the medical procedure. The information can include instructions that,when executed in conjunction with the processor 604, cause the basemodule 618 to control components of the operating room 602 in real-timeto obtain relevant measurements. Purely by way of example, theinstructions can cause the base module 618 control the hybrid MWPMcomponent 610 and/or perform actions related to testing and analysis(e.g., to perform a scan of the tissue sample, filter raw data receivedfrom the scan, analyze values from the scan, and the like). In someembodiments, the instructions cause the base module 618 to perform thescan with the hybrid MWPM component 610 repeatedly during a time frameand/or to repeat the scan after a predetermined time period. In someembodiments, the base module 618 includes a plurality of sub-modules. Invarious such embodiments, the plurality of sub-modules include aninitiation module configured to initiate the system 600; a pollingmodule configured to turn on and/or calibrate the hybrid MWPM component610; an analysis module configured to identify correlations betweenvalues from the scan and one or more reference cases, then identify asuitable reference case based on the correlations; a mapping moduleconfigured to map the values from the scan with the hybrid MWPMcomponent 610 with the imaging device data; and/or a communicationmodule configured to send raw data, processed data, and/or the resultsfrom analyses to the database 616. Additional details on thefunctionality of the base module 618, in accordance with someembodiments of the present technology, are described below withreference to FIGS. 8A and 8B.

In some embodiments, the operating room 602 and/or components thereofutilize artificial intelligence (AI) and/or Machine learning (ML) toanalyze the values from the scan, provide recommendations for the scan,and identify correlations between the data received from the hybrid MWPMcomponent 610 and the imaging device. Purely by way of example, theAI/ML can analyze the values from the scan (e.g., the raw data from thehybrid MWPM component 610), to provide recommendations related to theprocedure and/or tests of the tissue sample (e.g., to rescan a portionof the tissue sample, to scan an additional portion of the bone, and thelike). In another example, the AI/ML can identify correlations betweenvarious types of data to create a function to predict future eventsand/or measurements in the surgical procedure. For example, when thehybrid MWPM component 610 is directed toward a bone tissue and generatesa light beam with energy per laser pulse in a range between 15 mJ/cm² to20 mJ/cm², and a beam splitter divides the light beam in a 1:9 ratio,with a diameter of the light beam as 4 mm, and the values from the scanindicate a relative bone material density (BMD) value of 2.5 with amulti-wavelength photoacoustic measurement (MWPM) of 750 nm and a μ_(a)value of 2, the AI/ML data can predict the values that will be obtainedin scans of other tissue on the patient. Additionally, or alternatively,the AI/ML can identify reference cases with similar tissue compositionsbased on the values from the scan, imagery of the tissue sample, and/orother information received from the hybrid MWPM component 610 (e.g.,operational parameters during the scan). By generating correlationsusing the AI/ML component, the operating room 602 (and componentsthereof) can identify previously unrecognized trends, increase theaccuracy of the reference cases identified, and increase the accuracy ofdiagnoses of the tissue sample.

As further illustrated in FIG. 6 , the system 600 can include externalreal-time analysis equipment 622 (“external equipment 622,” such asoff-premise equipment, third party equipment, and the like) that iscommunicatively coupled to the operating room 602 via a cloud network624. For example, the communication interface 620 can be communicativelycoupled to the cloud network 624 to facilitate communication from theoperating room 602 to the external equipment 622. In some embodiments,the communication interface 620 includes a radio communication or otherwired or wireless communication. In some embodiments, the communicationinterface 620 includes a wired and/or a wireless network connection. Thecommunication interface 620, if wireless, may be implemented usingcommunication techniques such as Visible Light Communication (VLC),Worldwide Interoperability for Microwave Access (WiMAX), Long TermEvolution (LTE), Wireless Local Area Network (WLAN), Infrared (IR)communication, Public Switched Telephone Network (PSTN), Radio waves,and other communication techniques, known in the art.

In some embodiments, the external equipment 622 communicates with thecommunication interface 620 to provide one or more services to theoperating room 602. For example, the external equipment 622 cancommunicate with the operating room 602 to store the real-time data,provide data for potential reference cases, and/or to perform any of theanalyses discussed above. To do so, the external equipment 622 caninclude a base module 626 configured to perform operations in a similarmanner as the base module 618 of the operating room 602. Further, theexternal equipment 622 can include an external real-time equipmentdatabase 628 (“database 628”) configured to update the database 616 viathe cloud network 624 periodically and/or constantly. In variousembodiments, the database 628 can be divided into various sub-databasesto store different information related to the patient, such as medicalrecords for each patient in the reference cases (e.g., previoussurgeries, operations, and/or associated illness of the patient),locations of medical procedures, doctors involved in the medicalprocedures, and the like.

In the embodiment illustrated in FIG. 6 , the database 628 includes anexternal bone database 630 (“bone database 630”) and an externalpathology analysis database 632 (“pathology analysis database 632”). Thebone database 630 can store bone condition data (e.g., diagnoses ofnormal, cancerous, osteoporosis, and the like) for a plurality ofpatients, and/or information related to the test(s) performed on eachpatient (e.g., values from a scan using the hybrid MWPM component 610,DEXA results, X-ray results, and the like). For example, for patient 1,the information in the bone database 630 can include the BMD value fromat least one test performed on a bone sample from patient 1 (e.g., a BMDvalue of 4), the MWPM value provided by the hybrid MWPM component 610(e.g., an MWPM value of 700 nm), a threshold MWPM value for the patient(e.g., of 750 nm), and/or a diagnosis of the bone sample from patient 1(e.g., given the BMD, MWPM, and threshold values above, that patient 1has a normal bone). In some embodiments, the pathology analysis database632 stores data similar to the bone database 630 and/or data related tovarious pathologies, conditions, and/or diseases that are notnecessarily related to patients' bones. Purely by way of example, thedata in the pathology analysis database 632 can indicate that patient 1has inflamed tissue surrounding a specific bone sample that underwent aprevious medical operation (e.g., a surgery to remove a bacterialinfection from the bone). In another example, the data in the pathologyanalysis database 632 can include indications of various otherpathological conditions the patient has (e.g., cancer, diabetes, lupusand rheumatoid arthritis, hyperthyroidism, celiac disease, asthma,multiple sclerosis, and/or any other pathological condition).

FIG. 7 is a block diagram of the hybrid MWPM component 610communicatively coupled in the system 600 of FIG. 6 in accordance withsome embodiments of the present technology. In the illustratedembodiment, the hybrid MWPM component 610 includes an MWPM-specificprocessor 702 (“MWPM processor 702”) that is operatively coupled to alaser device 704, a laser controller 706, an ultrasound transducer 708,a needle hydrophone 710, a pre-amplifier 712, a digitizer 714, a memory716, and a communication interface 718. The hybrid MWPM component 610can employ a variety of these components to perform one or more scans(e.g., tests) of the tissue sample (e.g., measurements of bone density,bone matrix composition, and the like) and/or to help analyze resultsfrom the test(s). In some embodiments, the scans are conducted inreal-time using the hybrid MWPM component 610. In some embodiments, eachof the components of the hybrid MWPM component 610 is operativelycoupled to the power supply 608.

The MWPM processor 702 can facilitate programming of the plurality ofcomponents to perform their desired functions. For example, the MWPMprocessor 702 can be configured to control operation of the laser device704 via the laser controller 706 (e.g., by activating and/or modulatingthe intensity of the laser device 704) and record measurements resultingfrom the laser's interactions with the tissue sample. The MWPM processor702 can then retrieve information from the laser device 704, theultrasound transducer 708, and/or the needle hydrophone 710 (e.g.,intensity of the laser device 704, during operation, values related tophotoacoustic signals from the tissue sample, and the like) and directthe information to the operating room 602.

The laser device 704 can be configured to impart a light beam with apredetermined (or calibrated) laser intensity over a sample once thehybrid MWPM component 610 is turned on. In some embodiments, the laserdevice 704 is a neodymium-doped yttrium aluminum garnet (Nd:YAG) laser.In some embodiments, the light beam generated by the laser device 704 isdivided into two or more parts to help increase the accuracy of anymeasurement performed by the laser device 704. For example, a first partof the light beam (sometimes also referred to herein as a “referencebeam”) can be projected onto a black rubber (or other suitable material)and then recorded by the ultrasound transducer 708 for calibration ofsubsequent signal magnitude. A second part of the light beam (sometimesalso referred to herein as an “analysis beam”) can be projected onto thetissue sample. The reference beam can be used to calibrate the magnitudeof the light beam, while the analysis beam is imparted over the tissuesample to excite a photoacoustic signal from the tissue and/or togenerate optical measurements near the surface of the tissue.

In some embodiments, the laser controller 706 is configured to modulatethe laser intensity (e.g., magnitude) of the laser device 704 and/or thewavelength of the light beam. In some embodiments, the laser controller706 is configured to control the orientation, position, and/or movementof the laser device 704 during operation (e.g., to move the laser device704 during a scan). In some embodiments, the laser device 704 measures aμ_(a) value of the tissue sample to detect cracks or indentations overthe sample in addition to, or in alternative to, various photoacousticmeasurements.

In some embodiments, the hybrid MWPM component 610 employs theultrasound transducer 708 to measure and/or detect a material density ofthe tissue sample (e.g., to detect a bone material density (BMD)). Theultrasound transducer 708 is a sound-related sensor that sendselectrical signals to the tissue sample (e.g., the bone) that strike thetissue sample and revert back to the ultrasound transducer 708 and/oranother measuring device, such as the needle hydrophone 710. Theultrasound transducer 708 can further excite the photoacoustic signal,which is then measured by the needle hydrophone 710. Additionally, oralternatively, the ultrasound transducer 708 can be used to detectand/or identify irregularities in the tissue sample. For example, theultrasound transducer 708 can receive a signal from the MWPM processor702 to vibrate at a predefined frequency, thereby generating anddirecting sound waves into the tissue sample. The sound waves travelthrough the tissue sample (e.g., through the bone) and are redirectedback to the ultrasound transducer 708 (and/or the needle hydrophone 710)when they are incident on an irregularity. In various embodiments, theirregularity can include, cracks, holes, and/or inconsistency in theBMD. In some embodiments, the sound waves generated are reflected backto the ultrasound transducer 708 in the form of an echo signal. In suchembodiments, the ultrasound transducer 708 can convert the echo signalinto an electrical signal for the MWPM processor 702. The ultrasoundtransducer 708 can calculate the time interval between sending the soundwave into the sample and receiving the echo signal, send the timeinterval to the MWPM processor 702, and the MWPM processor 702 can usethe time interval to help evaluate the tissue sample.

In some embodiments, the needle hydrophone 710 includes a piezo-ceramichydrophone sensor that can provide ultrasonic field mapping withpinpoint access and high spatial resolution. In some embodiments, thehybrid MWPM component 610 includes multiple of the needle hydrophones710, each directed towards the tissue sample and configured to generatean ultrasonic field map. In some embodiments, the needle hydrophone 710performs functions related to analysis of the sample with decreasedsensitivity flatness and narrower directivity. In some embodiments, thephotoacoustic signal received by the needle hydrophone 710 is fed to thepre-amplifier 712 to amplify its magnitude. In some embodiments, thephotoacoustic signal is routed to the pre-amplifier 712 as an electricsignal via the MWPM processor 702. In other embodiments, the needlehydrophone 710 routes the photoacoustic signal directly to thepre-amplifier 712.

The digitizer 714 can be configured to convert analog signals intodigital signals before and/or after they are amplified. In someembodiments, the digitizer 714 converts the amplified photoacousticsignal into the real-time data to be stored in the database 616 and/orthe memory 716. Further, the memory 716 can store parameters related tothe test(s) of the tissue sample (e.g., operating parameters such as thewavelength of the lase, magnitude of the laser, frequency of theultrasound sound waves; the density, porosity, and/or matrix structureof the tissue sample; MWPM values; μ3 values; and the like). In someembodiments, the converted signals from the digitizer 714 are stored inthe memory 716, and then sent to the database 616 via the communicationinterface 718 for further analysis or use. In a specific, non-limitingexample, the memory 716 can store data that the laser device 704generates a light beam with an energy-per-laser-pulse in a range between15 millijoules per square centimeter (mJ/cm²) to 20 mJ/cm² with adiameter of about 4 millimeters (mm), and the beam splitter divides thelight beam into two components in 1:9 ratios (reference beam:analysisbeam). Additionally, or alternatively, the memory 716 can store valuesgenerated using the light beam above, such as BMD value of 2.5, an MWPMvalue of 750 nanometers (nm), and a μ_(a) value of 2.

The communication interface 718 can connect the hybrid MWPM component610 to external resources (e.g., via the internet), such as shared poolsof configurable resources and higher-level services that can be rapidlyprovisioned with minimal management effort. Additionally, oralternatively, the communication interface 718 can be communicativelycoupled to the operating room 602 via the communication interface 620(FIG. 6 ) to be communicatively coupled to the user interface 606, thememory unit 614, the base module 618, and/or the processor 604 forreal-time assistance in the operating room 602. Additionally, oralternatively, the communication interface 718 can be synchronized withthe database 616 (FIG. 6 ) to store information associated withoperating room 602. In some embodiments, the communication interface 718is referred to as an internal communication component that facilitatesinternal communications of the operating room 602, while the cloudnetwork 624 (FIG. 6 ) is referred to as an external communicationcomponent that facilitates communication between the external equipment622 and the operating room 602.

Referring back to FIG. 6 , the base module 618 of the operating room 602can be configured to retrieve information related to the analysis of thetissue sample from the database 616, the bone database 630, and/or thedatabases 628. Further, the base module 618 can be configured tocontrol, direct, and/or perform operation of the operating room 602 inreal-time.

FIGS. 8A and 8B are flow diagrams of a process 800 for operating theoperating room 602 of FIG. 6 in accordance with some embodiments of thepresent technology. The process 800 can be executed by the base module618, or other suitable component (e.g., the base module 626), to executeat least a part of a medical procedure to analyze and/or diagnose atissue sample (e.g., a user's bone). Below, FIGS. 8A and 8B areexplained in conjunction with references to FIGS. 6, 7, and 9-12 . Itwill be understood that, in some embodiments, one or more of thefunctions noted in the blocks may occur out of the order noted in thedrawings. For example, two blocks shown in succession in FIGS. 8A and 8Bcan be executed concurrently, or substantially concurrently. In anotherexample, blocks can sometimes be executed in an alternative order,depending upon the functionality involved. In yet another example, theblocks of the process 800 can be split for execution between two or morecomponents. In still further examples, one or more of the blocksdiscussed below can be omitted from the process 800 altogether. Inaddition, the process descriptions or blocks in flow charts should beunderstood as representing decisions made by a hardware structure suchas a state machine.

The process 800 begins at block 802 with receiving a request from asurgical site to schedule the surgical procedure for performingreal-time analysis of the tissue sample. In various embodiments, theprocess 800 can receive the request from the surgical site (e.g., at theoperating room 602) and/or a remote location. In some embodiments, therequest can include directions for scheduling a surgical procedure forperforming real-time analysis of a tissue sample. The directions caninclude the allocation of equipment (e.g., any of the components of theoperating room 602, ventilators, operation-specific equipment, and/orother equipment required at the surgical site) and personnel for thesurgical procedure. Purely by way of example, the request can include anindication that a patient (e.g., “Alex”) is being admitted at a surgicalfacility in New York, USA; that the operating room 602 needs to performsurgery with real-time analysis of bone tissue to help analyze anddiagnose Alex's bones, and that the surgical procedure will be conductedunder the supervision of Dr. Van. In some embodiments of this example,the base module 618 receives the request from the surgical facility inNew York to schedule the surgery for Alex.

At block 804, the process includes activating power for the operatingroom 602 via the power supply 608 in conjunction with the initiationmodule. In some embodiments, the initiation module performs functionsrelated to communication between the operating room 602 and the externalequipment 622. In some embodiments, the base module 618 communicateswith the processor 604 of the operating room 602 to turn on the powersupply 608. Further, at block 806, the process 800 includes establishingcommunication with the external equipment 622 to retrieve data relatedto previous tests for the patient (e.g., relevant values from Alex'sprevious surgical procedures), reference cases, and/or predeterminedmedical threshold values for one or more diagnosis. Purely by way ofexample, the base module 618 can retrieve, from the bone database 630,one or more predetermined threshold values associated with osteoporosis(e.g., that osteoporosis is considered for MWPM measurements with avalue range between about 700 nm and about 950 nm, a threshold value of750 nm or greater, and a BMD value of less than −2). In someembodiments, the process 800 executes block 806 using the initiationmodule within the base module 618. In various embodiments, theinitiation module can retrieve the data over the cloud network 624and/or any other suitable network connection.

At block 808, the process 800 includes updating the database 616 withthe data retrieved from the database 628. Purely by way of example, asillustrated in FIG. 9 , the database 616 can be updated with valuesassociated with one or more (four shown) measurements during a previoussurgical procedure. In the illustrated embodiment, for example, thedatabase 616 has been updated with values from 1:25:30 PM that includeBMD value of 2.5, an MWPM of 750 nm, and a μ_(a) value of 2; values from1:25:35 PM that include a BMD value of 0.9 with an MWPM of 780 nm, and aμ_(a) value of 2.5; values from 1:25:40 PM that include a BMD value of2.0 with an MWPM of 705 nm, and a μ_(a) value of 3; and values from1:25:45 PM that include a BMD value of 1.5 with an MWPM of 710 nm, and aμ_(a) value of 2. Each of the updates can also be associated with anobservation and/or diagnosis based on the measured values. For example,at 1:25:30 PM the values indicated high-density bone material, while at1:25:35 PM the values indicated porous bone material. Returning to thedescription of FIGS. 8A and 8B, in some embodiments, block 808 can beexecuted at least partially by the initiation module.

At block 810 the process 800 includes checking the calibration of thebeam splitter. In one embodiment, the base module 618 may split thelight beam generated by the laser device 704 into the reference beam andthe analysis beam using the beam splitter. For example, the base module618 can check that the beam splitter divides the light beam from thelaser device 704 (FIG. 7 ) in a 1:9 ratio (or other suitable ratio),that the diameter of the light beam is about 4 mm, and/or that the lightbeam has an energy-per-laser-pulse between about 15 mJ/cm² and about 20mJ/cm². In some embodiments, block 810 can be executed at leastpartially by the initiation module.

At block 812, the process 800 includes activating the hybrid MWPMcomponent 610 (FIG. 6 ), for example via the polling module of the basemodule 618. In some embodiments, the activation of the hybrid MWPMcomponent 610 to initiate testing of the sample by imparting theanalysis beam to illuminate the sample from one side at specificoperating parameters. Purely by way of example, the base module 618 canactivate the hybrid MWPM component 610 by setting theenergy-per-laser-pulse in the light beam from the laser device 704 at 15mJ/cm² with a broad bandwidth at 450 MHz, and instructing the hybridMWPM component 610 to begin emitting pulses. In some embodiments, block812 can be executed at least partially by the polling module.

In various embodiments, each sub-modules of the base module 618 (e.g.,the initiation module, the polling module, and the like) can executetheir functions above in blocks 804-812 concurrently and/or in asuccession alternative to the order discussed above. In someembodiments, the order of the execution of blocks 804-812 is at leastpartially dependent upon the input request received at the base module618 at block 802.

At block 814, the process 800 includes collecting data related to thetissue sample, from the hybrid MWPM component 610 and/or an imagingdevice. In some embodiments, the process 800 collects the dataperiodically in a predefined time frame. In various embodiments, thepredefined time frame can be set by the processor 604 of the operatingroom 602 (FIG. 6 ) according to the pulse width of the light beamemitted by the laser device 704, according to a movement speed of thelaser device 704 (e.g., a scan speed), a requested sample period, and/orany other suitable constraint. In a specific, non-limiting example, thebase module 618 can collect data every 1, 2, 5, 10, 50, and/or 100milliseconds (ms), and/or after any other suitable period. Returning tothe example of FIG. 9 , the predefined time frame in the retrieved datawas every five seconds. In some embodiments, block 814 can be executedat least partially by the polling module.

At block 816, the process 800 includes sending the collected data to thedatabase 616, along with any suitable related information. For example,the base module 618 can send data indicating that an MWPM value of 800nm was received from the tissue sample by illuminating one side of thetarget bone tissue with the energy-per-laser-pulse of 15 mJ/cm² from thelaser device 704. In some embodiments, block 816 can be executed atleast partially by the polling module.

Successively, at block 818, the process 800 includes generating one ormore correlations of real-time data from the surgical procedure (e.g.,Alex's tissue sample data) to one or more reference cases in the bonedatabase 630 e.g., tissue sample data for patients 1−N) to identify oneor more best match correlations. As discussed above, data from the bestmatch correlation(s) can be used to help analyze the real-time dataand/or to help diagnose the tissue sample. In some embodiments, the basemodule 618, using an AI/ML algorithm, generates the correlations betweenthe data received from the hybrid MWPM component 610 and the dataretrieved from the bone database 630 in real-time to identify one ormore closely related correlations in the reference cases. Purely by wayof a simplified example, the base module 618 can correlate the real-timedata acquired from the hybrid MWPM component 610 (e.g., an MWPM value of700 nm, a μ_(a) value of 2, and a BMD value of 2.5) with the retrieveddata from the bone database 630. In this example, the base module 618can identify reference cases with similar values from the scan, as wellas threshold values based on the similar cases (e.g., a threshold limitof 750 nm for osteoporosis based on the reference cases that arediagnosed with osteoporosis typically having an MWPM value below 750nm). Accordingly, in this example, the base module 618 can identify thatthe MWPM value of 700 nm is below the threshold limit of 750 nm forosteoporosis consideration and diagnose osteoporosis in the target bonetissue (e.g., despite the BMD value of 2.5). In some embodiments, block818 can be executed at least partially by the analysis module. Thethreshold limit for a condition can be selected or determined based onpatient information (e.g., age of the patient, gender, race, etc.). Forexample, threshold limits for osteoporosis consideration and diagnosiscan decrease with the patient's age to account for normal age-relatedbone-density decrease. Additionally, or alternatively, the thresholdlimits can be determined using machine-learning techniques disclosedherein.

The values from the scan can be used to provide a confidence score ofthe diagnosis, diagnose other conditions, etc. In some embodiments, thebase module 618 generates an aggregate score based on selected weightedvalues from the scan, then uses the aggregate score to characterize thetissue sample, diagnose conditions, or the like. The similar values inthe reference cases can be identified within ranges of values (e.g.,absolute ranges, percentage ranges, etc.), threshold values, and/ordetermined values using, for example, ML training sets, user input, etc.The system can be programmed with one or more correlation rules forconditions, patient groups, etc. In supervised training, a user canselect training sets of reference data in which pathology and accuracyof detected values have been validated. In unsupervised training, thesystem can select validated reference data sets and can be retrained anynumber of times.

At block 820, the process 800 includes determining a diagnosis of thetissue sample using the generated correlation(s), the identifiedreference cases, and/or any other suitable data from the bone database630 (e.g., diagnoses in the reference cases). In some embodiments, thebase module 618 determines the diagnosis based at least partially basedon diagnoses in the identified reference cases. For example, the basemodule 618 can mine data from the reference cases to determine that anMWPM value of 700 nm received from the tissue sample, with a BMD valueof 2.5, and a μ_(a) value of 2, exceeds a BMD threshold value of −2identified in the reference cases as a threshold for osteoporosis. As aresult, the base module 618 can diagnose that the bone tissue does notfall within the osteoporosis group. In some embodiments, block 820 canbe executed at least partially by the analysis module.

At block 822, the process 800 can include sending the diagnosis to thedatabase 616 and/or the user interface 606. For example, the base module618 can send an indication to the database 616 that the tissue sampledoes not fall within the osteoporosis group, along with an indication ofthe rationale (e.g., because the results indicated that the tissuesample had a BMD value above the threshold value) secondary support(e.g., that the tissue sample had an MWPM value below the thresholdlimit, imagery of the tissue sample corroborated the diagnosis, and thelike), and/or a record of the values from the scan and relatedinformation (e.g., operating parameters during the scan).

At block 824, the process 800 includes generating a 3D map of the datafrom the hybrid MWPM component 610 (e.g., the values from the scan,diagnostic information, and the like), and/or any linked data from theimaging device (example shown in FIG. 10 , discussed in more detailbelow). For example, the data from the imaging device can be mapped withthe hybrid MWPM component 610 using time stamps and/or metadataindicating the relative position and/or orientation of the imagingdevice and the hybrid MWPM component 610. In some embodiments, theimaging device and the hybrid MWPM component 610 are contained withinthe same end effector (e.g., when the imaging device 611 is used), andare therefore automatically linked. In some embodiments, the imagingdevice and the hybrid MWPM component 610 are positioned on exactopposite sides of the tissue sample (e.g., when a secondary imagingdevice is used), and are linked through a well-posed inversion. Invarious other embodiments, the process 800 uses the data on theorientation, position, and/or movement of each of the components of thehybrid MWPM component 610 during the scan to construct a 3D map withsimilar data on the orientation, position, and/or movement of theimaging device during the scan.

FIG. 10 illustrates an example of a 3D map 1000 in accordance with someembodiments of the present technology. The 3D map 1000 can allow thedoctor to view the real-time data in three dimensions to analyze thetissue sample concurrently with the process 800 and/or to reviewdiagnoses determined at block 820. In some embodiments, the process 800maps the diagnoses with the data of the imaging device to create the 3Dmap 1000 of the results for the doctor to analyze. In some embodiments,the 3D map 1000 highlights regions of the tissue sample according to thediagnoses and/or the values from the hybrid MWMP component 610. Forexample, the 3D map 1000 can highlight (or otherwise emphasize) regionswith BMD values that are within a particular range and/or MWPM valueswithin another particular range. In the illustrated embodiment, forexample, the 3D map 1000 emphasizes region 1020 via a change in shadingfrom the surrounding regions. The emphasis can help direct the doctor'sreview of the 3D map 1000 and/or further analysis of the patient. Insome embodiments, the process 800 creates the 3D map 1000 including rawdata from at least one of the BMD values and the MWPM values, allowingthe doctor to perform an independent diagnosis of the tissue sample. Invarious embodiments, the raw data can be indicated via differences inshading, color, fill patterns, express indications, display tables,selectable displays, and/or in any other suitable manner. In someembodiments, the 3D map 1000 includes selectable layers. For example,the 3D map 1000 can include a first layer created with the BMD values, asecond layer created with the MWPM values, and a third layer with datacorrelated from an imaging device. As further illustrated in FIG. 10 ,the selectable layers in the 3D map 1000 can be related to one or moreregions and/or depths within the bone. In some embodiments, block 824can be executed at least partially by the mapping module.

In some embodiments, the values from the scan, along with any identifiedreference cases, are sent to the auxiliary user interface for a doctor,surgeon, or other medical professional to perform real-time analysis ofthe data. In some such embodiments, the auxiliary interface alsodisplays data from the imaging device (e.g., in the form of a picture,image, and/or video). Accordingly, in such embodiments, the doctor,surgeon, or other medical professional can conclude a final diagnosisbased on the values from the scan, the reference cases, diagnoses fromblock 820, and/or the data captured by the imaging device. Purely by wayof example, the process 800 can create the 3D map, with selectablelayers showing that the BMD value ranges from 1.5 to 4 through thetissue sample, with an average (or mean) value above the threshold valueof 2; the MWPM value ranges from 500 nm to 900 nm, with an average (ormean) value below the threshold limit of 750 nm for the osteoporosisgroup; the data from the imaging device indicating high density in thestructure of the bone (e.g., supporting the average BMD value above thethreshold value); and the diagnosis from block 820 of a normal bone. Inthis example, the doctor, surgeon, or other medical professional mayconclude that the bone tissue does not have osteoporosis, but may wantto further review regions with an MWPM value below 750 nm. For example,the mapped data may indicate a range of values suggesting that minorosteoporosis and/or a transition state from a healthy bone in a fewregions that can be identified by the doctor while reviewing all of thedata together. Accordingly, the generated 3D map can help increase earlyidentification of bone regions with pathological conditions (e.g.,before the conditions become widespread).

At block 826, the process 800 includes sending the data from the scan,the diagnosis from block 820, the 3D map, and/or the diagnosis from thedoctor (or other medical professional) to the bone database 630. Forexample, the base module 618 can send information indicating the BMDvalue is 2.5, the threshold value for BMD was set at 2, the MWPM valueis 700 nm, the threshold value for MWPM was set at 750 nm, the capturedimage supporting the real-time data, and/or a confirmation of thedoctor's diagnosis of the bone. In one embodiment, the base module 618updates the bone database 630 over the cloud network 624. In someembodiments, block 826 can be executed at least partially by thecommunication module.

The base module 626 of the external equipment 622 can be configured toretrieve, from the database 616, information related to the analysis ofthe tissue sample. Further, as noted above, the base module 626 can beconfigured to perform the operation of the operating room 602 inreal-time in accordance with the process 800 discussed above.

Additionally, or alternatively, the base module 626 can be configured tocontrol the external equipment 622 in real-time in conjunction with theprocesses in the operating room 602. FIG. 11 is a flow diagram of aprocess 1100 transferring information to the external equipment 622 inaccordance with some embodiments of the present technology. The process1100 is described in conjunction with the bone database 630 illustratedin FIG. 12 and the pathology analysis database 632 illustrated in FIG.13 . FIG. 11 is also explained in conjunction with various details fromFIGS. 6-9 . It will be understood that, in some embodiments, one or moreof the functions noted in the blocks may occur out of the order noted inthe drawings. For example, two blocks shown in succession in FIG. 11 canbe executed concurrently, or substantially concurrently. In anotherexample, blocks can sometimes be executed in an alternative order,depending upon the functionality involved. In yet another example, theblocks of the process 1100 can be split for execution between two ormore components. In still further examples, one or more of the blocksdiscussed below can be omitted from the process 1100 altogether. Inaddition, the process descriptions or blocks in flow charts should beunderstood as representing decisions made by a hardware structure suchas a state machine

The process 1100 begins at block 1102 by establishing a connection withthe operating room 602 to retrieve information related to the hybridMWPM component 610. In some embodiments, the base module 626 facilitatesconnection to the operating room 602 (e.g., via the cloud network 624and/or another suitable network), to retrieve data related to the hybridMWPM component 610 from the database 616. In one example, the retrievedinformation can include operating parameters the hybrid MWPM component610 and/or recent calibration data related to the hybrid MWPM component610. In another example, the base module 626 can retrieve data that aspecific patient (e.g., Alex) is being evaluated for bone-relatedconditions and is going to be tested for osteoporosis (or relatedpathologies) using the hybrid MWPM component 610. In this example, thebase module 626 can retrieve the predetermined thresholds (e.g., the BMDthreshold value is 2 and/or the MWPM threshold value is 950 nm) and/orthe real-time data for Alex's bone tissue (e.g., Alex's BMD value is 2.5and/or Alex's MWPM value is 900 nm, each indicating no osteoporosis).Additionally, or alternatively, the base module 626 can retrievecalibration data related to hybrid MWPM component 610. Purely by way ofexample, the calibration data can include post-measurement adjustments,such as lowering the MWPM value by 2 nm for correction.

At block 1104, the process 1100 includes updating the database 628. Insome embodiments, the process 1100 allows researchers to update the bonedatabase 630 (example shown in FIG. 12 , discussed in more detail below)and/or the pathology analysis database 632 (example shown in FIG. 13 ,discussed in more detail below) according to the diagnosis and/or valuesfrom the tissue sample (e.g., Alex's raw values and Alex's bonediagnosis). Additionally, or alternatively, researchers (or otherappropriate external groups such as a board of medical professionals)can update the bone database 630 and/or the pathology analysis database632 with data from tests conducted over a plurality of tissue samplesand/or patients, along with the corresponding diagnoses from a doctor orother medical professional. In some embodiments, the plurality of testsare experimental and/or conducted to gather data, rather than forclinical purposes. In a specific example, the plurality of tests canprovide reference cases with a variety of operating parameters for thehybrid MWPM component 610. In another example, the process 1100 allowsthe researchers to update the bone database 630 with an analysis ofnumerous samples conducted using the hybrid MWPM component 610 atspecific operational settings. In some embodiments, the data resultsfrom a clinical trial. Additionally, or alternatively, the data can beincluded to increase the diversity of observational and/or training datafor the AI/ML. In each case, the updates to the bone database 630 and/orthe pathology analysis database 632 increase the body of potentialreference cases that can be identified at block 818 of FIG. 8B. In someembodiments, block 1104 is executed by a research module in the basemodule 626.

FIG. 12 illustrates the bone database 630 in accordance with someembodiments of the present technology. As illustrated in FIG. 12 , thebone database 630 is configured to store data related to the analysis ofthe tissue sample for multiple patients, such as the diagnosis for eachpatient and/or the data related to that diagnosis. In some embodiments,the bone database 630 includes one or more subsets of grouped and/orpaired data sets of data (e.g., all patients with a given BMD value, allpatients with a given diagnosis, and the like). Further, the bonedatabase 630 can store specific data points (e.g., the MWPM value alongwith descriptions from the imaging device) for each patient, along witha diagnosis of the tissue sample, such as osteoporosis and/or normalbone. In the illustrated example, the bone database 630 stores that fora first region of patient 1, the imaging device recorded high densitystructures; the hybrid MWPM component 610 measured an MWPM value of 730nm and a BMD value of 2.2; and that the tissue sample was diagnosed asnormal bone. In another example, the bone database 630 stores that for asecond region of patient 1, the imaging device recorded porousstructures; the hybrid MWPM component 610 measured an MWPM value of 780nm and a BMD value of 1.8; and that the tissue sample was diagnosed ashaving osteoporosis.

FIG. 13 illustrates the pathology analysis database 632 in accordancewith some embodiments of the present technology. In the illustratedembodiment, the pathology analysis database 632 is configured to storedata related to the analysis of the tissue sample, bone tissuecondition, inflamed tissue surrounding the target tissue, cancerouscells, and the like. Further, as illustrated in FIG. 13 , the pathologyanalysis database 632 can store data associated with a plurality ofpatients, each of whom underwent some form of testing on their bodilytissue for pathological conditions and/or diseases. In some embodiments,the pathology analysis database 632 stores data related to the sampleusing the pathology component 612 and the measurements detected and/ormonitored over the imaging device. The data can be related to diagnosedconditions of the monitored tissue samples, such as the patient havingskin cancer. In some embodiments, the data is related to the analysis ofthe tissue sample includes a range of measurements and threshold valuesthat the pathology component 612 received while measuring a tissuesample. In some embodiments, the threshold values indicate a maximum anda minimum value of data collected by the pathology component 612 whileperforming the analysis. In the illustrated example, the pathologyanalysis database 632 stores that for a patient 2, the pathologycomponent 612 detected symptoms of skin cancer and the imaging devicerecorded melanoma and brown spots on patient 2's skin.

Returning to FIG. 11 , at block 1106 the process 1100 includesperforming an informational handshake with the operating room 602 toreceive a transfer of data from the operating room 602. In someembodiments, the process 1100 performs the handshake continuously withthe operating room 602 to receive constant updates to the bone database630 and the pathology analysis database 632. In some embodiments, theprocess 1100 performs the handshake with the operating room 602 onlyonce (e.g., concurrently with block 826 of FIG. 8B). In someembodiments, block 1106 is executed by an initiation module in the basemodule 626.

At block 1108, the process 1100 updates the database 628 by extractingthe values from the hybrid MWPM component 610 and/or diagnostic datafrom the database 616. For example, the base module 626 can update thebone database 630 with the values from the scan (e.g., the raw data fromthe hybrid MWPM component 610) and/or the analysis performed on thetissue sample using raw data. In a specific example, the update to thebone database can include recording measurements of the patient's BMDvalue of 2.5, the MWPM value of 900 nm, and a related diagnosis is thatthe bone tissue does not have osteoporosis.

In some alternate embodiments, the surgical procedure, and any updatesto relevant databases, is fully autonomous and executed by the operatingroom 602. The automation can allow for measurements and analysis withoutthe input and/or intervention of a doctor. As a result, the automationcan increase access to the processes disclosed herein, thereby helpingto detect bone conditions early. In some embodiments, the surgicalprocedures described herein are fully controlled manually by the doctorin conjunction with the hybrid MWPM component 610 and/or the pathologycomponent 612. In such embodiments, the doctor's use of the hybrid MWPMcomponent 610 can help increase the accuracy of their diagnosis and/orallow them to provide a real-time diagnosis (e.g., instead of waitingfor biopsy results). In some embodiments, the surgical proceduresdescribed herein are executed via a hybrid process, where a few partsare manually controlled and other parts are completed autonomously orsemi-autonomously (e.g., with the doctor acting as a check to the hybridMWPM component 610). In some such embodiments, a surgical robotutilizing the data from the bone database 630 can propose an action forreview by the doctor, surgeon, or other medical professional. If theaction is approved, the hybrid MWPM component 610 can execute theaction. In some semi-autonomous embodiments, the hybrid MWPM component610 performs the functions of analyzing bone tissue using real-time datafrom the bone database 630 of the external equipment 622, then presentsthe diagnoses to the doctor, surgeon, or other medical professional forreview.

It will be appreciated by those skilled in the art that changes could bemade to the exemplary embodiments described above without departing fromthe broad inventive concept thereof. It is to be understood, therefore,that this disclosure is not limited to the particular embodimentsdisclosed, but it is intended to cover modifications within the spiritand scope of the subject disclosure as disclosed above.

Examples

The present technology is illustrated, for example, according to variousaspects described below. Various examples of aspects of the presenttechnology are described as numbered examples (1, 2, 3, etc.) forconvenience. These are provided as examples and do not limit the presenttechnology. It is noted that any of the dependent examples can becombined in any suitable manner, and placed into a respectiveindependent example. The other examples can be presented in a similarmanner.

In some examples systems, the system can (1) perform multiplemeasurements, (2) determining a set of measurements corresponding totargeted tissue, (3) determining correlations between measurements inthe set and/or between the set of measurements and reference data set,and (4) generating one or more multi-modality outputs (e.g., compositeimage, score, report) for the targeted tissue based on the correlations.The system can perform scans for obtaining the measurements. The systemcan identify when the device is located at target position for analyzingthe tissue.

1. A computer-implemented method for real-time mapping and analysis ofbone tissue, the method comprising:

-   -   receiving, from a bone tissue mapper device, patient data of a        patient,        -   wherein the bone tissue mapper device includes a hybrid            multi-wavelength photoacoustic measurements (MWPM) unit, and        -   wherein the received patient data includes one or more            images of at least one target tissue of the bone tissue and            at least one measurement of the at least one target tissue            from the hybrid MWPM unit;    -   correlating, using a computing system, the at least one        measurement to a plurality of stored reference cases;    -   identifying, using the computing system, one or more reference        cases, from the plurality of stored reference cases, based on        one or more correlations between the at least one measurement        received from the hybrid MWPM component and previous        measurements in each the plurality of stored reference cases;    -   determining, using the computing system and based on the one or        more identified reference cases, a diagnosis of a bone condition        of the at least one target tissue;    -   generating, using the computing system, a diagnostic map of the        bone tissue based on the image data and the diagnosis of the        bone condition of the at least one target tissue; and    -   sending, using the computing system, the diagnostic map to a        computing device accessible by a surgeon.

2. The method of example 1, wherein the diagnostic map includes athree-dimensional (3D) bone map of a bone in the at least one targettissue, and wherein the method further comprises:

-   -   analyzing the 3D bone map using a machine learning model trained        using bone tissue training sets to determine at least one        surgical step based on the 3D bone map; and    -   performing, using a robotic surgery system, the at least one        surgical step on the patient.

3. The method of any of examples 1 and 2 wherein the at least onemeasurement includes a measurement of one or more of: bone mineraldensity, an μ3 value, and a MWPM value.

4. The method of any of examples 1-3 wherein each of the plurality ofreference cases includes measurements from a related hybrid MWPMcomponent, one or more other measurements related to an associatedpatient, and diagnostic information for the associated patient.

5. The method of any of examples 1-4 wherein the correlations areidentified using an artificial intelligence or machine learning (AI/ML)algorithm.

6. The method of any of examples 1-5 wherein the at least onemeasurement is at least one first measurement received at a first time,and wherein the method further comprises:

-   -   receiving at least one second measurement of the at least one        region of the patient from the hybrid MWPM component at a second        time, wherein identifying the one or more reference cases is        further based on correlations between the at least one second        measurement and previous measurements of the plurality of        reference cases.

7. The method of any of examples 1-6 wherein the diagnosis of the bonecondition includes one or more of: osteoporosis, clinical osteopenia,bone cancer, normal bone with low or high bone mineral density (BMD),and/or osteomyelitis.

8. The method of any of examples 1-7 wherein sending the diagnosis ofthe bone condition to a computing device includes sending an indicationof the correlations between the at least one measurement and theprevious measurements for review by the surgeon.

9. A system for performing real-time analysis of bone tissue during asurgical procedure, the system comprising:

-   -   one or more computer processors; and    -   a non-transitory computer-readable storage medium storing        computer instructions, which when executed by the one or more        computer processors, cause the system to:        -   receive at least one first measurement from a hybrid            multi-wavelength photoacoustic measurements (MWPM)            component, the at least one first measurement indicative of            a first bone material density in a tissue sample of a            patient;        -   receive image data of the tissue sample from an imaging            device, wherein the imaging device is spatially coupled to            the MWPM component;        -   retrieve a plurality of reference cases, wherein each            reference case is associated with an individual reference            patient, and wherein each reference case includes at least            one second measurement indicative of a second bone material            density in a tissue sample of the individual reference            patient;        -   identify one or more reference cases from the plurality of            reference cases based on correlations between the at least            one first measurement and the at least one second            measurement in each reference case;        -   determine, based on the one or more identified reference            cases, at least one diagnosis of a bone condition in the            tissue sample of the patient;        -   generating a diagnostic map of the tissue sample based on            the image data and the at least one diagnosis of the bone            condition in the tissue sample of the patient; and        -   display, on a user interface accessible by a surgeon before            or during a surgical operation on the patient, the            diagnostic map.

10. The system of example 9, further comprising a network interface,wherein the computer instructions further cause the system to retrievethe plurality of reference cases from a third party database via thenetwork interface.

11. The system of any of examples 9 and 10 wherein the diagnostic mapincludes a three-dimensional (3D) map the tissue sample of the patientbased at least in part on the at least one first measurement and theimage data.

12. The system of example 11 wherein the at least one first measurementis two or more first measurements, wherein the 3D map includes two ormore selectable layers, and wherein each of the selectable layers isassociated with a corresponding one of the two or more measurements.

13. The system of any of examples 11 and 12 wherein the at least onediagnosis of the bone condition is at least two diagnoses of the bonecondition, wherein the 3D map includes two or more selectable layers,and wherein each of the selectable layers is associated with anindividual one of the at least two diagnoses.

14. The system of any of examples 11-13, further comprising the imagingdevice, wherein the 3D map is further based at least in part on imagedata from the imaging device.

15. The system of any of examples 9-14, wherein the computerinstructions further cause the system to:

monitor, through the imaging device, one or more specific aspects ofbone tissue in the tissue sample of the patient based on diagnosticinformation in the one or more identified reference cases.

16. The system of example 15 wherein the one or more specific aspects ofbone tissue include at least one of porous structures and high densitystructures.

17. A computer-implemented method for performing real-time analysis ofbone tissue during a surgical procedure, the computer-implemented methodcomprising:

-   -   receiving, from a hybrid multi-wavelength photoacoustic        measurements (MWPM) component, at least one measurement        associated with tissue sample of a bone mass in a patient;    -   receiving, from an imaging device, image data related to the        tissue sample, wherein the imaging device is coupled to the        hybrid MWPM component to image the tissue sample from a single        position;    -   identifying one or more reference patients from a plurality of        reference cases based on correlations between the at least one        measurement and previous measurements associated with previous        tissue samples of bone mass in each of the plurality of        reference cases;    -   determining, based on the one or more identified reference        cases, a diagnosis of at least one bone condition of the        patient;    -   generating a diagnostic map of the tissue sample based on the        image data and the diagnosis of the at least one bone condition        of the patient; and    -   sending the diagnosis to a computing device accessible by a        medical professional.

18. The computer-implemented method of example 17 wherein the hybridMWPM component includes a laser device configured to emit a light beamused in performing the at least one measurement, and wherein the methodfurther comprises:

-   -   splitting the light beam, using a beam splitter, into a first        portion directed at the bone mass in the patient and a second        portion directed at a calibration material;    -   receiving, from the hybrid MWPM component, at least one        calibration measurement associated the second portion of the        laser energy; and    -   adjusting the at least one measurement associated with the bone        mass based on the at least one calibration measurement before        identifying the one or more reference patients.

19. The computer-implemented method of any of examples 17 and 18 whereinthe hybrid MWPM component includes a neodymium-doped yttrium aluminumgarnet laser.

20. The computer-implemented method of any of examples 17-19 whereindetermining the diagnosis includes determining whether a metric in theat least one measurement exceeds a threshold value.

21. The computer-implemented method of example 20 wherein the thresholdvalue is set based on the previous measurements each of the one or moreidentified reference cases.

22. The computer-implemented method of examples 20 and 21, furthercomprising retrieving the threshold value from an external database.

23. A computer-implemented method for real-time mapping and analysis oftarget tissue, the method comprising:

-   -   receiving, from a tissue mapping device, patient data of a        patient, wherein:        -   the tissue mapping device includes a hybrid imaging            component configured to take two or more wavelength-based            measurements of the target tissue, and        -   the received patient data includes one or more images of the            target tissue and at least two or more wavelength-based            measurements of the target tissue;    -   correlating, using a computing system, the two or more        wavelength-based measurements to a plurality of stored reference        cases;    -   identifying, using the computing system, one or more reference        cases, from the plurality of stored reference cases, based on        one or more correlations between the two or more        wavelength-based measurements received from the hybrid imaging        component and previous measurements in each the plurality of        stored reference cases;    -   determining, using the computing system and based on the one or        more identified reference cases, a diagnosis of the target        tissue;    -   generating, using the computing system, a diagnostic map of the        target tissue based on the image data and the diagnosis of the        target tissue; and    -   sending, using the computing system, the diagnostic map to a        computing device accessible by a surgeon.

24. The compute-implemented method of example 23 wherein the patientdata is received during a surgical operation on a patient, and whereinthe diagnostic map is displayed on the computing device during thesurgical operation on the patient.

25. The compute-implemented method of any of examples 23 and 24 whereinthe two or more wavelength-based measurements include at least one of anultrasound image, a multiwavelength photoacoustic measurement, a μ3measurement, an x-ray image, and a computerized tomography scan.

26. The compute-implemented method of any of examples 23 and 24 whereinthe two or more wavelength-based measurements include at least a firstx-ray image taken at a first wavelength and a second x-ray image takenat a second wavelength different than the first wavelength.

27. A system for performing real-time analysis of bone tissue during asurgical procedure, the system comprising:

-   -   one or more computer processors; and    -   a non-transitory computer-readable storage medium storing        computer instructions, which when executed by the one or more        computer processors, cause the system to perform any of the        actions of examples 23-26.

CONCLUSION

From the foregoing, it will be appreciated that specific embodiments ofthe technology have been described herein for purposes of illustration,but well-known structures and functions have not been shown or describedin detail to avoid unnecessarily obscuring the description of theembodiments of the technology. To the extent any material incorporatedherein by reference conflicts with the present disclosure, the presentdisclosure controls. Where the context permits, singular or plural termsmay also include the plural or singular term, respectively. Moreover,unless the word “or” is expressly limited to mean only a single itemexclusive from the other items in reference to a list of two or moreitems, then the use of “or” in such a list is to be interpreted asincluding (a) any single item in the list, (b) all of the items in thelist, or (c) any combination of the items in the list. Furthermore, asused herein, the phrase “and/or” as in “A and/or B” refers to A alone, Balone, and both A and B. Additionally, the terms “comprising,”“including,” “having,” and “with” are used throughout to mean includingat least the recited feature(s) such that any greater number of the samefeatures and/or additional types of other features are not precluded.Further, the terms “approximately” and “about” are used herein to meanwithin at least within 10 percent of a given value or limit. Purely byway of example, an approximate ratio means within a ten percent of thegiven ratio.

From the foregoing, it will also be appreciated that variousmodifications may be made without deviating from the disclosure or thetechnology. For example, one of ordinary skill in the art willunderstand that various components of the technology can be furtherdivided into subcomponents, or that various components and functions ofthe technology may be combined and integrated. In addition, certainaspects of the technology described in the context of particularembodiments may also be combined or eliminated in other embodiments.

Furthermore, although advantages associated with certain embodiments ofthe technology have been described in the context of those embodiments,other embodiments may also exhibit such advantages, and not allembodiments need necessarily exhibit such advantages to fall within thescope of the technology. Accordingly, the disclosure and associatedtechnology can encompass other embodiments not expressly shown ordescribed herein.

We claim:
 1. A computer-implemented method for analyzing target tissue,the method comprising: receiving, from a tissue mapping device, at leastone image of the target tissue and at least two wavelength-basedmeasurements of the target tissue, wherein the tissue mapping devicecomprises a hybrid imaging component; identifying at least one referencecase based on the at least two wavelength-based measurements;determining, based on the at least one identified reference case, adiagnosis of the target tissue; generating a diagnostic map of thetarget tissue based on the at least one image and the diagnosis; andsending the diagnostic map to a computing device.
 2. The method of claim1, wherein the at least two wavelength-based measurements comprise atleast one of an ultrasound image, a multiwavelength photoacousticmeasurement, a μ3 measurement, an x-ray image, or a computerizedtomography scan.
 3. The method of claim 1, wherein the at least twowavelength-based measurements comprise at least a first x-ray imagetaken at a first wavelength and a second x-ray image taken at a secondwavelength different from the first wavelength.
 4. The method of claim1, wherein identifying the at least one reference cases comprises:extracting features from the at least two wavelength-based measurements,the features comprising at least a bone material density; and providingthe at least one reference case based on the features, the machinelearning model trained to generate reference cases based onmulti-wavelength photoacoustic measurements (MWPMs).
 5. The method ofclaim 1, wherein the diagnostic map includes a three-dimensional (3D)bone map of a bone in the target tissue, the method comprising:analyzing the 3D bone map using a machine learning model trained usingbone tissue training sets to determine at least one surgical step; andperforming, using a surgical robot, the at least one surgical step. 6.The method of claim 5, wherein the diagnosis comprises at least twodiagnoses of a bone condition, wherein the 3D map includes at least twoselectable layers, and wherein each of the at least two selectablelayers is associated with an individual one of the at least twodiagnoses.
 7. The method of claim 1, wherein the diagnosis comprises atleast one of osteoporosis, clinical osteopenia, bone cancer, normal bonewith low or high bone mineral density (BMD), or osteomyelitis.
 8. Asystem for analyzing target tissue, the system comprising: one or morecomputer processors; and a non-transitory computer-readable storagemedium storing computer instructions, which when executed by the one ormore computer processors, cause the system to: receive, from a tissuemapping device, at least one image of the target tissue and at least twowavelength-based measurements of the target tissue, wherein the tissuemapping device comprises a hybrid imaging component; identify at leastone reference case based on the at least two wavelength-basedmeasurements; determine, based on the at least one identified referencecase, a diagnosis of the target tissue; generate a diagnostic map of thetarget tissue based on the at least one image and the diagnosis; andsend the diagnostic map to a computing device.
 9. The system of claim 8,wherein the at least two wavelength-based measurements comprise at leastone of an ultrasound image, a multiwavelength photoacoustic measurement,a μ3 measurement, an x-ray image, or a computerized tomography scan. 10.The system of claim 8, wherein the at least two wavelength-basedmeasurements comprise at least a first x-ray image taken at a firstwavelength and a second x-ray image taken at a second wavelengthdifferent from the first wavelength.
 11. The system of claim 8 whereinthe computer instructions to identify the at least one reference casecause the system to: extract features from the at least twowavelength-based measurements, the features comprising at least a bonematerial density; and provide the at least one reference case based onthe features, the machine learning model trained to generate referencecases based on multi-wavelength photoacoustic measurements (MWPMs). 12.The system of claim 8, wherein the diagnostic map includes athree-dimensional (3D) bone map of a bone in the target tissue, and thecomputer instructions to identify the at least one reference case causethe system to: analyze the 3D bone map using a machine learning modeltrained using bone tissue training sets to determine at least onesurgical step; and perform, using a surgical robot, the at least onesurgical step.
 13. The system of claim 12, wherein the diagnosiscomprises at least two diagnoses of a bone condition, wherein the 3D mapincludes at least two selectable layers, and wherein each of the atleast two selectable layers is associated with an individual one of theat least two diagnoses.
 14. The system of claim 8, wherein the diagnosiscomprises at least one of osteoporosis, clinical osteopenia, bonecancer, normal bone with low or high bone mineral density (BMD), orosteomyelitis.
 15. A surgical robot for analyzing target tissue, thesurgical robot configured to: receive, from a tissue mapping device, atleast one image of the target tissue and at least two wavelength-basedmeasurements of the target tissue, wherein the tissue mapping devicecomprises a hybrid imaging component; identify at least one referencecase based on the at least two wavelength-based measurements; determine,based on the at least one identified reference case, a diagnosis of thetarget tissue; generate a diagnostic map of the target tissue based onthe at least one image and the diagnosis; and send the diagnostic map toa computing device.
 16. The surgical robot of claim 15, comprising ahybrid MWPM component comprising a laser device configured to emit alight beam used in performing the at least two wavelength-basedmeasurements, and wherein the surgical robot is configured to: split thelight beam, using a beam splitter, into a first portion directed at bonemass in the patient and a second portion directed at a calibrationmaterial; receive, from the hybrid MWPM component, at least onecalibration measurement associated with the second portion of the lightbeam; and adjust the at least two wavelength-based measurementsassociated with the bone mass based on the at least one calibrationmeasurement before identifying the at least one reference patient. 17.The surgical robot of claim 16, wherein the hybrid MWPM componentincludes a neodymium-doped yttrium aluminum garnet laser.
 18. Thesurgical robot of claim 15, wherein the surgical robot is configured todetermine the diagnosis by determining whether a metric in the at leastone measurement exceeds a threshold value.
 19. The surgical robot ofclaim 15, wherein the at least two wavelength-based measurementscomprise at least one of an ultrasound image, a multiwavelengthphotoacoustic measurement, a μ3 measurement, an x-ray image, or acomputerized tomography scan.
 20. The surgical robot of claim 15,wherein the at least two wavelength-based measurements comprise at leasta first x-ray image taken at a first wavelength and a second x-ray imagetaken at a second wavelength different from the first wavelength.